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Preface 


This  monograph  presents  observations  from  an  ongoing  research  proj¬ 
ect  cosponsored  by  the  Office  of  the  Assistant  Secretary  of  the  Navy 
for  Research,  Development,  and  Acquisition,  Chief  Systems  Engineer 
(ASN  RDA  CHSENG).  It  discusses  existing  policies  that  prevent  or 
inhibit  military  services  from  conducting  information  assurance  certi¬ 
fication  and  accreditation  for  a  collection  of  systems  that  are  colocated 
or  operate  on  a  common  platform. 

This  research  was  conducted  within  the  Acquisition  and  Technol¬ 
ogy  Policy  Center  of  the  RAND  National  Defense  Research  Institute, 
a  federally  funded  research  and  development  center  sponsored  by  the 
Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Com¬ 
batant  Commands,  the  Navy,  the  Marine  Corps,  the  defense  agencies, 
and  the  defense  Intelligence  Community. 

For  more  information  on  RAND’s  Acquisition  and  Technology 
Policy  Center,  contact  the  Director,  Philip  Anton.  He  can  be  reached 
by  email  at  atpc-director@rand.org;  by  phone  at  310-393-0411,  exten¬ 
sion  7798;  or  by  mail  at  the  RAND  Corporation,  1776  Main  Street, 
P.O.  Box  2138,  Santa  Monica,  California  90407-2138.  More  informa¬ 
tion  about  RAND  is  available  at  www.rand.org. 
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Summary 


The  challenges  associated  with  securing  U.S.  Department  of  Defense 
(DoD)  information  systems  (ISs)  have  grown  as  the  department’s  infor¬ 
mation  infrastructure  has  become  more  complex  and  interconnected. 
At  the  same  time,  the  potential  negative  consequences  associated  with 
cyber  intrusions  have  become  more  severe,  as  demonstrated  by  the 
recently  publicized  breach  of  computer  networks  at  defense  contractors 
involved  in  the  development  of  the  F-35  aircraft  (Gorman,  Cole,  and 
Dreazen,  2009).  An  important  question  to  consider  is  whether  cur¬ 
rent  information  assurance  (lA)  policies  and  procedures  are  sufficient 
to  address  this  growing  threat  and  well  suited  to  address  vulnerability 
issues  associated  with  highly  networked  ISs. 

Presently,  all  DoD  ISs  must  individually  satisfy  the  certifica¬ 
tion  and  accreditation  (C&A)  requirements  outlined  in  DoD  Instruc¬ 
tion  (DoDI)  8510.01,  “DoD  Information  Assurance  Certification  and 
Accreditation  Process  (DIACAP)”  (2007),  prior  to  receiving  authoriza¬ 
tion  to  operate  (ATO).  As  written,  the  DIACAP  is  focused  on  conduct¬ 
ing  C&A  for  a  single  system. 

As  the  number  of  individual  DoD  ISs  continues  to  grow,  and  as 
they  become  more  interdependent  and  are  integrated  in  more  complex 
ways  (for  example,  using  service-oriented  architectures,  or  SOAs),  the 
time  and  resources  required  to  complete  the  C&A  process  will  also 
increase.  Similarly,  the  current  C&A  process,  which  focuses  on  the 
individual,  discrete  IS,  may  overlook  potential  vulnerabilities  intro¬ 
duced  at  the  interface  between  an  ever-increasing  number  of  ISs  and 
by  increasingly  complex  network  connections.  Therefore,  DoD  might 
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find  it  necessary  to  consider  new  policies  and  procedures  for  assess¬ 
ing  lA  C&A  for  heterogeneous  and  variable  collections  of  networked 
systems  and  components.  The  objective  of  this  study  was  to  determine 
whether  there  were  any  existing  DoD  or  other  federal  policies  that 
could  prevent  or  inhibit  the  U.S.  Department  of  the  Navy  from  apply¬ 
ing  the  DIACAP  to  an  aggregate  of  ISs  or  systems  of  systems  (SoSs) 
that  are  colocated  or  operate  on  a  common  platform  (e.g.,  Navy  vessel 
or  aircraft).  A  revised  C&A  process  that  focuses  on  aggregates  of  ISs  or 
SoSs  should  ideally  provide  the  transparency  and  situational  awareness 
sought  by  the  current  process,  require  fewer  resources  to  conduct,  and 
identify  potential  vulnerabilities  that  exist  at  the  interface  between  ISs. 

We  considered  three  levels  of  aggregation.  The  first  was  the  full 
aggregation  approach  (option  1),  in  which  every  DoD  IS  on  the  plat¬ 
form  or  at  the  location  is  aggregated  into  a  single  DoD  IS.  The  second 
was  the  partial  aggregation  approach  (option  2),  in  which  systems  are 
logically  aggregated  such  that  the  final  number  of  aggregate  DoD  ISs 
is  less  than  the  original  number  of  ISs.  For  the  purposes  of  this  policy 
analysis,  we  aggregated  DoD  ISs  by  mission  assurance  category  (MAC), 
confidentiality  level  (CL),  and  mission  criticality  (MC).'  We  used  these 
categories  because  of  their  relationship  to  the  required  lA  controls  and 
the  final  accreditation  determination.  Further  investigation  would 
be  needed  to  determine  the  optimal  set  of  categories  for  aggregating 
DoD  IS.  The  final  case  that  we  investigated  involved  no  aggregation 
(option  3),  or  what  is  essentially  the  current  status  quo  defined  in  fed¬ 
eral  policy  documents,  in  which  each  system  is  assessed  and  certified 
individually.  The  final  analysis  for  each  of  the  three  types  of  aggrega¬ 
tion  is  shown  in  Table  S.l. 

The  partial  aggregation  approach  (option  2)  identified  fewer 
potential  policy  issues  and  fewer  implementation  difficulties  compared 
to  the  full  aggregation  approach.  Many  of  the  issues  associated  with 
implementing  a  partial  aggregation  approach  could  be  addressed 
with  minor  changes  to  the  current  DIACAP  System  Identification 
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See  Appendix  B  for  definitions  of  these  three  characteristics  and  their  levels. 
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Table  S.1 

Assessment  of  Policy  Issues  Related  to  IS  Aggregation 


Full  Partial  No 

Aggregation  Aggregation  Aggregation 


Degree  of  Aggregation 


Option  7 


Option  2 


Option  3 


1.  Initiate  and  plan  lA  C&A 

-Register  system  with  DoD 
component  lA  program 


-Assign  lA  controls 


-Assemble  DIACAPteam 

-Initiate  DIACAP  implementation 
plan 

2.  Implement  and  validate  assigned 
lA  controls 

-Execute  DIACAP  implementation 
plan 

-Conduct  validation  activities 

-Prepare  POA&M 

-Compile  validation  results  and 
DIACAP  Scorecard 

3.  Make  certification  determination 
and  accreditation  decisions 

-Make  certification  determination 

-Issue  accreditation  decision 

4.  Maintain  authorization  to  operate 
and  conduct  reviews 

-Maintain  situational  awareness 

-Maintain  lA  posture 

-Conduct  review  (at  least  annually) 

-Initiate  reaccreditation 

5.  Decommission 
-Retire  system 


I  No  policy  issues  identified 
□  No  policy  issues  identified;  potential 

difficulties  with  implementation  identified 
I  Potential  policy  issue(s)  identified 
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Profile  (SIP)  and  the  DIACAP  Scorecard.  It  would  also  be  necessary 
to  work  with  the  White  House’s  Office  of  Management  and  Budget 
(OMB)  to  determine  the  appropriate  level  of  aggregation  to  meet  OMB’s 
Plan  of  Action  and  Milestones  (POA&M)  reporting  requirements. 

Under  the  current  DoDI  8510.01,  lA  managers  encounter  dif¬ 
ficult  obstacles  associated  with  monitoring  lA  situational  awareness, 
conducting  lA  control  validation  activities,  summarizing  validation 
results,  and  attempting  to  preserve  the  lA  posture  of  their  systems  indi¬ 
vidually  and  collectively  as  part  of  a  larger  SoS.  The  difficulty  asso¬ 
ciated  with  these  activities  would  likely  persist  even  if  an  aggregate 
DoD  IS  approach  were  implemented  unless  new  standards  for  measur¬ 
ing  IS  security  are  developed,  along  with  new  techniques  for  moni¬ 
toring,  tracking,  and  validating  lA  controls.  These  techniques  should 
leverage  methods  derived  from  systems  engineering. 

We  identified  one  potential  policy  issue  for  this  approach  that 
would  require  significant  modification  to  DoD  policy.  Specifically, 
DoD  policy  does  not  currently  allow  for  the  decommissioning  or  the 
modification  of  a  portion  of  a  DoD  IS.  It  would  be  necessary  to  alter 
existing  policy  to  allow  a  component  DoD  IS  that  is  part  of  a  larger 
aggregate  DoD  IS  to  be  decommissioned  or  modified  without  the  need 
to  also  decommission  or  modify  the  larger  aggregate  DoD  IS.  Simi¬ 
larly,  there  is  no  method  to  verify  the  validity  or  accuracy  of  the  C&A 
assessment  for  a  DoD  IS  with  a  component  DoD  IS  that  has  been 
decommissioned,  modified,  or  removed.  Currently,  the  only  option  is 
to  recertify  the  entire  IS. 

In  the  Navy,  identical  or  nearly  identical  individual  ISs  are  imple¬ 
mented  across  different  platforms.  According  to  current  lA  policy,  each 
instantiation  of  an  IS  should  be  certified  and  accredited  independently. 
The  current  approach  is  possibly  justified  by  the  fact  that  the  configu¬ 
ration  of  individual  ISs  may  vary  across  platforms.  It  should  be  noted, 
however,  that  this  heterogeneity  potentially  introduces  lA  vulnerabili¬ 
ties  and  complexity.  Furthermore,  the  current  approach  may  cause  the 
Navy  to  incur  greater  costs  for  the  many  individual  lA  certifications 
required  than  if  a  common  configuration  of  individual  ISs  were  defined 
and  maintained  across  the  Navy  fleet  and  other  platforms.  Analysts  in 
the  Navy  and  in  DoD  have  started  to  develop  concepts  and  approaches 
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for  defining  common  secure  or  trusted  configurations  for  individual 
ISs.  Such  a  configuration  can  generally  be  characterized  as  the  lA  pedi¬ 
gree  of  an  IS.  Several  definitions  of  lA  pedigree  have  been  proposed. 
However,  in  order  for  the  concept  of  lA  pedigree  to  be  applied  effec¬ 
tively  to  lA  C&A  aggregation  efforts,  a  precise  definition  is  needed. 

Based  on  our  analysis  of  existing  policies,  we  make  the  following 
recommendations  to  enable  an  SoS  approach  to  conducting  lA  C&A: 

•  Policy  recommendations: 

-  Restructure  the  SIP  and  the  DIACAP  Scorecard  described  in 
DoDI  8510.01  to  allow  them  to  track  both  component  DoD 
ISs  and  aggregate  DoD  ISs. 

-  In  consultation  with  OMB,  develop  an  acceptable  level  of  DoD 
IS  aggregation,  and  develop  a  strategy  for  tracking  information 
security  performance  between  the  POA&M  and  DoD  budget 
documentation. 

-  Develop  or  adopt  a  common  set  of  IS  security  metrics  that  can 
be  used  to  aggregate  information  assurance  control  validation 
results  across  the  full  range  of  ISs. 

-  Develop  specific  guidance  and  policy  for  modifying  or  decom¬ 
missioning  components  or  subsystems  of  an  aggregated  DoD 
IS. 

•  Implementation  recommendations: 

-  Conduct  a  pilot  project  to  investigate  alternative  approaches  to 
and  categories  for  partial  aggregation  and  to  assess  the  poten¬ 
tial  benefits  of  lA  controls  and  C&A  procedures  for  an  aggre¬ 
gated  DoD  IS  or  DoD  SoS. 

-  Develop  and  refine  a  definition  of  IS  lA  pedigree  that  can  be 
used  in  the  lA  aggregation  C&A  process. 

In  this  monograph,  we  define  an  IS  lA  pedigree  as  including  an  IS 
configuration  management  plan  and  an  IS  lA  control  profile,  as  well 
as  other  lA  metrics.  (For  a  detailed  definition  of  an  IS  lA  pedigree,  see 
Chapter  Four  of  this  monograph.) 

Drawing  from  experience  in  other  areas  of  systems  engineering, 
it  is  possible  that  an  SoS  approach  to  lA  may  improve  overall  lA  per- 
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formance  and  enhance  overall  information  security  situational  aware¬ 
ness,  lA  posture,  and  overall  performance.  However,  this  has  yet  to  he 
proven.  Based  on  our  initial  analysis,  a  partial  aggregation  strategy  that 
used  MAC,  CL,  and  MC  as  the  principal  categories  for  aggregation 
appears  to  present  a  reasonable  first  approach  for  achieving  an  aggre¬ 
gated  C&A  process  and  would  require  relatively  few  changes  to  the 
current  process  outlined  in  DoDI  8510.01. 

The  current  DIACAP  process  has  been  characterized  as  a  signifi¬ 
cant  improvement  over  its  predecessor.  However,  it  is  not  without  its 
own  limitations.  As  DoD  and  the  rest  of  the  federal  government  move 
toward  a  more  decentralized,  service- oriented  architecture,  the  pro¬ 
cess  of  conducting  lA  C&A  will  become  more  daunting,  and  an  ever- 
increasing  number  of  potentially  critical  lA  vulnerabilities  will  likely 
go  unidentified  until  it  is  too  late.  Therefore,  it  is  important  for  DoD  to 
investigate  systems  engineering  methods  and  techniques  to  help  ensure 
the  protection  and  availability  of  the  nation’s  critical  communication 
and  information  networks. 
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CHAPTER  ONE 


Background  and  Objective 


Background 

The  purpose  of  information  assurance  (lA)  is  to  protect  information 
systems  (ISs)  from  serious  and  frivolous  attacks  and  to  ensure  the  integ¬ 
rity,  availability,  confidentiality,  authentication,  and  nonrepudiation  of 
the  information  contained  therein.  Over  time,  there  has  been  a  steady 
increase  in  the  number  of  attacks  against  federal  agencies’  computer 
systems,  including  those  of  the  U.S.  Department  of  Defense  (DoD) 
(Nakashima,  2008).  DoD  has  disclosed  to  Congress  that  it  detected 
360  million  attempts  to  penetrate  its  networks  in  2008 — up  from 
6  million  in  2006 — and  that  it  had  spent  $100  million  in  the  previous 
six  months  repairing  damage  from  cyberattacks  (Gorman,  Cole,  and 
Dreazen,  2009). 

The  challenges  associated  with  securing  DoD  ISs  have  grown  as 
the  department’s  information  infrastructure  has  become  more  exten¬ 
sive,  complex,  and  interconnected.  At  the  same  time,  the  potential  neg¬ 
ative  consequences  associated  with  cyber  intrusions  have  also  become 
more  severe,  as  demonstrated  by  the  recently  publicized  breach  of  com¬ 
puter  networks  at  defense  contractors  involved  in  the  development  of 
the  F-35  aircraft  (Gorman,  Cole,  and  Dreazen,  2009).  While  contra¬ 
dictory  statements  have  been  issued  regarding  the  intrusion  into  the 
F-35  program  and  whether  the  unclassified  networks  of  primary  U.S. 
defense  contractors  were  compromised  (Nakashima,  2009;  Fulghum 
and  Warwick,  2009),  it  is  clear  that  the  U.S.  government  has  made  sig¬ 
nificant  changes  in  response  to  this  and  related  intrusions. 
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In  early  2007,  the  Air  Force  launched  a  partnership  with  about  a 
dozen  contractors  working  on  the  F-35  and  F-22  programs.  In  August 
of  that  year,  then-Deputy  Secretary  of  Defense  Gordon  England  gath¬ 
ered  the  top  executives  of  major  contractors  for  a  classihed  hriehng  on 
cyher  threats  to  their  companies: 

“We  shared  with  them  the  fact  that  we’ve  got  a  very,  very  aggres¬ 
sive  cyber  threat,”  said  Robert  Lentz,  a  Pentagon  official  who 
heads  the  partnership.  The  Pentagon  soon  will  seek  to  amend 
defense  acquisition  rules  to  require  cybersecurity  standards  for 
firms  seeking  contracts.  (Nakashima,  2009) 

Other  changes  to  DoD  lA  policy  are  also  under  consideration, 
including  revisiting  the  DoD  certification  and  accreditation  (C&A) 
framework.  This  framework  is  critical  for  ensuring  that  lA  controls 
and  procedures  for  securing  DoD  ISs  are  in  compliance  with  DoD  and 
federal  policies. 

An  important  question  to  consider  is  whether  current  lA  policies 
and  procedures  are  sufficient  to  address  the  threat.  Most  current  lA 
policies  and  procedures  are  focused  on  protecting  and  securing  indi¬ 
vidual  ISs  and  the  networks  that  connect  them.  However,  they  do  not 
identify  or  address  vulnerabilities  that  can  emerge  when  ISs  and  net¬ 
works  that  are  certified  and  accredited  individually  are  interconnected 
in  more  complex  configurations. 

Although  small  portions  of  the  overall  DoD  network  (e.g.,  enclaves 
and  local  area  networks)  have  been  certified  using  the  DIACAP  pro¬ 
cess,  DoD  has  not  attempted  to  apply  this  C&A  process  collectively 
to  all  system  types  or  all  systems  in  the  entire  Global  Information 
Grid  (GIG).  This  is  probably  not  technically  feasible  for  a  number  of 
reasons,  in  particular  because  the  GIG  is  a  constantly  changing  and 
growing  network  of  networks.'  In  addition,  the  advent  of  Web  ser- 


*  While  much  greater  configuration  control  is  maintained  over  the  GIG  than  over  the 
Internet,  the  two  collections  of  networks  have  many  similarities,  especially  in  terms  of 
configuration-control  challenges,  because  of  the  many  hosts  and  interconnections  that  char¬ 
acterize  both  networks,  the  many  heterogeneous  IT  standards  in  use,  and  the  many  genera¬ 
tions  of  technologies  employed.  This  lack  of  homogeneity  presents  significant  I A  challenges. 
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vices  and  service- oriented  architectures  (SOAs)  has  further  increased 
the  interconnectivity  of  DoD  and  federal  ISs.  SOAs  are  becoming  the 
standard  design  paradigm  for  new  and  emerging  DoD  ISs.  The  rapid 
evolution  of  interconnected  ISs  and  new  SOA  technologies  suggest  that 
a  larger  collection  of  ISs  should  undergo  lA  C&A  as  a  configuration- 
controlled  collection  of  systems. 

Another  important  question  is  whether  the  lA  C&A  frame¬ 
work  can  accommodate  the  growing  number  and  complexity  of  ISs 
being  installed  on  military  platforms  and  at  fixed  locations.  In  addi¬ 
tion,  there  is  the  question  of  whether  the  time  necessary  to  execute  the 
C&A  process  makes  it  less  effective — both  operationally  and  from  a 
security  perspective. 

In  a  recent  speech,  General  Peter  Chiarelli,  the  U.S.  Army  Vice 
Chief  of  Staff,  touted  the  rapid  development  and  deployment  of  two 
new  ISs  that  have  provided  U.S.  forces  with  important  new  capa¬ 
bilities  during  recent  operations  in  Iraq  (Chiarelli,  2009).  He  had 
a  major  hand  in  bringing  these  systems  to  the  field  without  going 
through  the  traditional  lA  certification  process.^  Both  were  developed 
by  the  Defense  Advanced  Research  Projects  Agency  outside  of  the 
DoDI  5000  acquisition  process  (see  DoDI  5000.02,  2008).  In  justify¬ 
ing  the  reasons  for  this.  General  Chiarelli  characterized  the  lA  cer¬ 
tification  process  as  broken  and  stated  that,  if  the  Army  had  had  to 
put  these  systems  through  the  traditional  lA  certification  process. 
Command  Post  of  the  Future  would  only  now  be  reaching  troops  in 
the  field,  in  2009.  And,  even  though  the  Tactical  Ground  Reporting 
System  completed  initial  development  in  2008,  by  the  end  of  2009, 
more  than  19  brigade  combat  teams  will  be  equipped  with  this  new  IS 
(Chiarelli  2009). 

The  Navy  faces  similar  challenges.  It  takes  on  average  18  months 
to  complete  an  I A  C&A  for  a  typical  IS  on  a  Navy  ship  (Newborn, 
2009).  Navy  cruisers  have  upwards  of  27  separate  networks  onboard, 
and  carriers  have  up  to  44  separate  networks  (Kiriakou,  2009).  In  addi- 


^  The  systems  are  Command  Post  of  the  Future  and  the  Tactical  Ground  Reporting  System. 
The  first  was  fielded  in  2005,  less  than  a  year  after  its  initial  development.  The  latter  was 
fielded  in  2008  after  a  similarly  compressed  development  schedule. 
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tion,  each  ship  has  hundreds  of  individual  ISs,  each  requiring  lA  C&A. 
These  lA  C&A  processes  are  conducted  in  parallel,  frequently  using 
independent  test  regimes  or  different  IS  vulnerability  and  exposure 
enumerations,  even  though  the  ISs  will  he  operated  simultaneously  in 
a  mutually  interdependent  way  during  actual  operations.  Increasingly, 
in  the  Navy  and  at  the  joint  program  level,  a  system  of  systems  (SoS) 
approach  is  being  used  to  manage  the  development  of  large  system 
clusters.  Therefore,  it  is  useful  to  consider  whether  an  SoS  approach 
can  improve  effectiveness  and  reduce  the  time  required  for  lA  DoD 
C&A  processes. 


Objective 

Presently,  all  DoD  ISs  must  satisfy  the  C&A  requirements  outlined  in 
DoD  Instruction  (DoDI)  8510.01  “DoD  Information  Assurance  Cer¬ 
tification  and  Accreditation  Process  (DIACAP)”  (2007)  prior  to  receiv¬ 
ing  authorization  to  operate  (ATO).^  Current  DoD  and  federal  poli¬ 
cies  are  defined  for  individual,  homogeneous  systems  or  components. 
However,  as  the  number  of  individual  DoD  ISs  continues  to  grow,  and 
as  these  systems  become  more  interdependent  and  are  integrated  in 
more  complex  ways  (for  example,  using  SOAs),  the  time  and  resources 
required  to  complete  the  C&A  process  will  also  increase.  Similarly, 
the  current  C&A  framework,  which  focuses  on  the  individual,  discrete 
IS,  may  overlook  potential  vulnerabilities  introduced  at  the  interface 
between  an  ever-increasing  number  of  ISs  and  by  increasingly  com¬ 
plex  network  connections.  Therefore,  DoD  might  find  it  necessary  to 
consider  new  policies  and  procedures  to  assess  lA  C&A  for  hetero¬ 
geneous  collections  of  networked  systems  and  components  that  vary 
across  multiple  dimensions  (e.g.,  mission  assurance  category  [MAC], 


^  DoD  IS  is  defined  in  DoDI  8500.2  (2003)  as  the  “[s]et  of  information  resources  organized 
for  the  collection,  storage,  processing,  maintenance,  use,  sharing,  dissemination,  disposition, 
display,  or  transmission  of  information.  Includes  AIS  [automated  information  system]  appli¬ 
cations,  enclaves,  outsourced  IT-based  processes,  and  platform  IT  interconnections”  (para. 
E2.1.17).  It  includes  within  its  definition  hardware,  software,  and  networks. 
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confidentiality  level  [CL],  mission  criticality  [MC],  function,  life-cycle 
phase). 

The  objective  of  this  study  was  to  determine  whether  there  are  any 
existing  DoD  or  other  federal  policies  that  would  prevent  or  inhibit  the 
U.S.  Department  of  the  Navy  from  applying  the  DIACAP  to  an  aggre¬ 
gate  of  ISs  or  SoS  that  are  colocated  or  operate  on  a  common  platform 
(e.g..  Navy  vessel  or  aircraft).  We  also  examined  whether  the  current 
single-system  C&A  approach  can  be  extended  with  minimal  changes 
to  DoD  or  U.S.  government  policy  to  address  lA  vulnerabilities  that 
are  evident  only  in  the  SoS  context.  Ideally,  a  revised  C&A  frame¬ 
work  should  provide  all  the  benefits  sought  by  the  current  process  but 
consume  fewer  resources,  not  impair  the  operational  readiness  of  the 
vessel,  and  identify  vulnerabilities  that  can  arise  from  the  integration 
of  individual  systems. 


Organization  of  This  Monograph 

This  monograph  is  organized  as  follows.  Chapter  One  has  briefly 
explored  the  issues  concerning  the  need  for  lA  C&A  and  the  moti¬ 
vation  for  this  study.  Chapter  Two  discusses  some  of  the  challenges 
associated  with  the  current  framework  as  applied  to  DoD  informa¬ 
tion  infrastructure.  Chapter  Three  provides  a  brief  introduction  to  the 
DIACAP  process  and  a  description  of  its  primary  activities. 

Chapter  Four  introduces  the  concept  of  degrees  of  aggregation 
of  DoD  ISs  for  the  purposes  of  C&A  and  provides  an  assessment  of 
specific  DIACAP  activities  that  either  inhibit  or  preclude  aggregated 
C&A  because  of  current  DoD  or  federal  policies.  That  chapter  also 
includes  a  discussion  of  some  of  the  potential  security  trade-offs  and 
offers  a  definition  of  IS  lA  pedigree  for  use  in  the  lA  aggregation  C&A 
process. 

Chapter  Five  concludes  the  monograph  with  preliminary  recom¬ 
mendations  for  changes  to  DoD  and  federal  policies  to  enable  the  cur¬ 
rent  DIACAP  to  be  applied  to  C&A  of  an  aggregation  of  DoD  ISs  in 
a  logically  defined  SoS.  It  also  proposes  some  additional  considerations 
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and  strategies  as  the  Department  of  the  Navy  moves  forward  with  SoS 
C&A  aggregation. 

Finally,  two  appendixes  provide  background  on  the  DIACAP 
System  Identihcation  Prohle  (SIP)  and  some  of  the  terminology  used 
in  this  monograph. 


CHAPTER  TWO 


Growing  Challenges  for  the  Information 
Assurance  Certification  and  Accreditation  of 
DoD  Information  Systems 


DoD  acquisition  programs  that  produce  ISs  face  growing  challenges 
in  obtaining  lA  C&A.  In  this  chapter,  we  review  some  of  the  factors 
that  appear  to  influence  these  trends  and  that  are  increasing  the  time 
required  to  obtain  lA  C&A. 


Software  Complexity 

The  first  of  these  factors  is  the  growing  software  complexity  of  DoD 
ISs.  The  number  of  software  lines  of  code  in  commercial,  off  the  shelf 
(COTS)  systems  is  growing  significantly.  Figure  2.1  shows  the  growth 
in  the  size  of  two  important  elements  of  the  COTS  software  infrastruc¬ 
ture,  the  Microsoft®  Windows®  and  Debian  Linux  operating  systems 
(OSs). 

The  Microsoft  Windows  OS  has  grown  exponentially  in  the  past 
decade  and  now  contains  more  than  50  million  lines  of  code.  What  is 
even  more  surprising  is  the  large  size  of  Debian  Linux,  one  particular 
variant  of  the  open-source  Linux  OS.  This  software  code  base  now 
comprises  more  than  200  million  lines  of  code.  As  the  functionality 
of  these  two  OSs  has  grown  over  the  past  decade,  their  complexity  has 
also  increased  significantly. 

Of  course,  DoD  is  not  immune  to  these  trends.  DoD  ISs,  includ¬ 
ing  weapon  systems,  are  increasing  in  their  software  complexity  as  well. 
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Figure  2.1 

Source  Lines  of  Code  for  the  Windows  and  Debian  Linux  Operating 
Systems 


(1993)  (1994)  (1993)  (2000)  (2007) 

Operating  system  (year) 
SOURCE:  Defense  Science  Board  (2009,  p. 
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13,  Figure  5). 


As  the  Defense  Science  Board  noted  in  its  recent  report  on  infor¬ 
mation  technology  (IT)  acquisition  issues, 


Software  has  spread  well  beyond  defense  infrastructure  into  the 
very  heart  of  weapon  systems.  For  example,  thousands  of  micro¬ 
processors,  linear  electric  drive  controllers,  dynamic  sensors,  and 
millions  of  lines  of  sophisticated  code  enable  the  startling  capa¬ 
bilities  of  the  F-22  and  Joint  Strike  Fighter,  as  well  as  quantum 
increases  in  the  sensitivity  achieved  using  pre-existing  sensors. 
Several  years  ago  a  handheld  grenade  launcher  was  created  with 
smart  projectiles  guided  by  2,000  lines  of  code.  Moreover,  the 
software  code  base  within  mission  systems  is  growing  rapidly 
from  generation  to  generation.  (Defense  Science  Board,  2009, 
p.  14) 


In  DoD  systems,  executable  source  lines  of  code  (ESLOC)  are 
one  metric  for  measuring  the  size  and  complexity  of  a  piece  of  soft¬ 
ware.  This  metric  is  independent  of  the  source  code  associated  with 
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the  COTS  base  or  platform  on  which  a  particular  system  may  be 
built.  Figure  2.2  illustrates  ESLOC  growth  over  time  for  a  selection  of 
weapon  systems  and  how  it  is  expected  to  dramatically  increase  out  to 
the  year  2020. 

The  IS  component  of  the  U.S.  Navy  DDG-1000  program  con¬ 
tains  almost  10  million  ESLOC,  more  than  a  50-percent  increase  from 
the  Virginia  SSN  program  and  an  even  larger  increase  from  the  Aegis 
7.1R  baseline. 


Increasing  Software  Vulnerabilities  and  Malware 
Population 

Growing  software  complexity  and  size  increases  the  possibility  for 
intentionally  hidden  functions  to  go  unnoticed  (Baker,  2007).  Perhaps 
more  importantly,  unintentional  errors  could  remain  undetected  in  the 
code  (Lyons,  2007).  It  is  difficult  to  produce  flawless  code  with  no 
errors.  Software  defects  can  reduce  the  effectiveness  of  security  features 

Figure  2.2 

Executable  Source  Lines  of  Code  for  Selected  Weapon  Systems 
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and  can  be  exploited  by  an  adversary  to  gain  root-level  access  to  an  IS. 
Studies  of  software  defects  in  large  code  bases  indicate  that  a  defect 
rate  of  one  per  thousand  lines  of  code  is  typical  (Marchenko  and 
Abrahamsson,  2007).  This  implies  that  there  may  have  been  approxi¬ 
mately  50,000  defects  in  the  Microsoft  Windows  Vista  OS  at  its  ini¬ 
tial  release,  and  more  200,000  defects  in  the  initial  release  of  Debian 
Linux.  It  also  implies  that  there  may  be  10,000  defects  in  the  Navy’s 
DD(X)  software  code  base.  These  are  I A  vulnerabilities  that  state- 
sponsored  adversaries  or  hackers  could  potentially  take  advantage  of 

In  addition,  a  greater  amount  of  software  is  being  produced  over¬ 
seas  in  countries  with  lower  software  engineering  costs.  These  countries 
may  also  harbor  organizations  and  individuals  with  ulterior  motives  for 
producing  software  with  flaws  of  one  sort  or  another.  Recent  data  indi¬ 
cate  that  more  than  70  percent  of  firms  in  the  U.S.  software  industry 
are  now  offshoring,  or  producing  major  software  components  overseas 
(Defense  Science  Board,  2009).  These  trends  raise  concerns  about  the 
level  of  trust  that  can  be  placed  in  a  software  supply  chain  that  now 
extends,  in  most  cases,  beyond  the  United  States,  increasing  the  need 
for  rigorous  and  comprehensive  analysis  of  software  code  bases  used  in 
DoD  ISs. 

Another  important  factor  that  influences  system  complexity  and 
the  need  for  comprehensive  lA  C&A  processes  is  the  growing  threat 
that  DoD  systems  face  from  malware.  Earlier  in  this  monograph,  we 
described  some  of  the  most  visible  and  publicly  discussed  threats  that 
have  affected  DoD  programs.  Another  measure  of  the  threat  is  the 
number  of  viruses  or  distinct  malware  programs  being  released  over 
the  Internet  that  affect  both  commercial  and  DoD  systems.  The 
number  of  viruses  has  increased  exponentially  over  time  and  is  dou¬ 
bling  on  a  year-to-year  basis.  In  2007,  roughly  half  a  million  viruses 
were  released  over  the  Internet,  requiring  antivirus  programmers  to 
produce  a  large  number  of  signatures  and  countermeasures.  In  2008, 
the  number  of  viruses  released  into  the  “wild”  reached  about  a  million 
and  2009  numbers  appear  to  be  on  track  to  reach  2  million  (Viega, 
2009).  What  are  the  implications  for  lA  C&A?  The  C&A  process  must 
demonstrate  that  the  IS  in  question  is  not  vulnerable  to  all  known 
viruses,  for  example.  With  the  virus  population  increasing  so  rapidly. 
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test  scripts  are  also  growing  in  size,  and  tests  must  be  run  repeatedly  to 
keep  up  with  the  number  of  viruses,  resulting  in  an  increase  in  the  size 
and  complexity  of  lA  tools  and  tests. 


Limitations  of  Automated  Software  Review  Tools 

While  automated  tools  exist  to  examine  the  integrity  of  software  code, 
these  tools  are  far  from  perfect  and,  in  many  cases,  can  be  used  only 
to  examine  code  written  in  specific  software  languages.  This  implies 
that  highly  trained  individuals  are  needed  to  examine  software  code 
for  flaws  and  to  ensure  its  integrity.  The  time  needed  to  review  large 
software  programs  is  one  significant  factor  in  determining  the  overall 
amount  of  time  required  for  a  program  to  obtain  lA  C&A.  For  exam¬ 
ple,  when  the  initial  version  of  the  Joint  Tactical  Radio  System  (JTRS) 
wideband  networking  waveform  was  completed,  it  was  submitted  to 
the  National  Security  Agency  for  lA  C&A.  In  late  2006,  it  was  deter¬ 
mined  that  this  software  code  contained  2.4  million  lines.  Initially, 
the  JTRS  program  office  had  hoped  that  this  software  review  could 
be  completed  within  three  months.  Experts  at  the  National  Security 
Agency  were  able  to  determine  very  early  on  that  three  months  would 
be  an  insufficient  amount  of  time  to  review  software  of  this  complexity* 


Challenge  of  Incremental  Program  Development 

A  third  factor  that  makes  it  difficult  to  obtain  clear-cut  lA  C&A 
decisions  is  that  IS  programs  are  increasingly  utilizing  an  incremen¬ 
tal  development  or  spiral  development  approach.  The  JTRS  program 
again  provides  a  good  example.  The  wideband  networking  waveform 
development  effort  has  proceeded  over  a  number  of  years.  Roughly 
every  six  months,  a  new  version  of  the  waveform  has  been  developed 
to  try  to  meet  the  operational  needs  of  the  program.  In  other  words. 


*  Personal  communication  from  Jarrat  Mowery,  deputy  technical  director  of  the  JTRS 
Joint  Program  Executive  Office. 
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the  code  base  for  this  IS  has  changed  significantly  over  time.  However, 
a  program  code  base  must  be  frozen  before  it  can  undergo  a  definitive 
lA  certification.  This  means  that  lA  certification  of  software  code  has 
to  be  done  serially,  after  the  program  has  completed  development  of  a 
major  software  version,  or  more  than  once  in  incrementally  developed 
programs.  Incremental  development  of  IS  programs  has  become  the 
norm  rather  than  the  exception  in  DoD  in  recent  years. 


Increasing  Scrutiny  of  Programs 

A  fourth  reason  that  lA  certification  has  become  more  difficult  is  that 
the  level  of  scrutiny  of  programs  has  increased  significantly.  This  was  as 
a  result,  at  least  in  part,  of  a  retrospective  review  that  determined  that 
a  number  of  older,  legacy  programs  have  serious  lA  vulnerability  flaws 
that  were  not  detected  during  program  development.  A  thorough  eval¬ 
uation  of  this  problem  is  beyond  the  scope  of  this  monograph,  however. 


System  Interdependence  and  Interconnectedness 

Other  reasons  for  the  growing  difficulty  associated  with  lA  C&A  have 
to  do  with  the  growing  interdependence  and  interconnectedness  of 
ISs,  both  in  the  open  commercial  market  and  in  DoD.  Indeed,  the 
entire  concept  of  the  GIG  is  based  on  the  ability  of  ISs  to  be  intercon¬ 
nected  regardless  of  their  function  or  organizational  lineage.  It  should 
also  be  remembered  that  the  GIG  vision  includes  not  just  the  ability 
of  communication  systems  to  be  linked  together,  but  also  that  com¬ 
puting  resources  should  be  shared  and  made  available  dynamically  to 
users  regardless  of  their  physical  location.  The  growing  interoperability 
afforded  by  the  GIG  also  introduces  potential  vulnerabilities  that,  in 
the  past,  may  have  affected  only  one  small  component  of  the  GIG  but 
may  now  lead  to  increased  vulnerability  across  the  entire  network. 
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Configuration  Management  and  System  Administration 

Configuration  management  is,  itself,  a  growing  challenge  as  systems 
become  more  complex.  It  is  often  possible  to  secure  a  system  by  care¬ 
fully  managing  its  configuration  and  ensuring  that  all  ports  and  pro¬ 
tocols  are  set  to  a  specific  set  of  conditions  or  options  that  eliminate 
vulnerabilities.  However,  as  the  complexity  of  systems  grows,  so  do 
the  number  of  options  or  settings  that  must  be  managed  in  these  sys¬ 
tems.  To  ensure  the  security  and  integrity  of  the  system,  highly  trained 
operators  may  be  needed  to  prevent  the  settings  from  being  changed 
inadvertently  or,  if  they  are  changed,  to  ensure  that  they  do  not  intro¬ 
duce  potential  vulnerabilities. 

A  related  challenge  is  the  administration  of  systems  of  greater 
capability  and  functionality.  Even  a  desktop  computer  today  has 
much  greater  functionality  and  associated  management  complexity  at 
the  enterprise  level  than  a  desktop  computer  from  several  IT  genera¬ 
tions  ago.  While  automated  tools  have  been  introduced  to  help  system 
administrators  manage  networks  of  PCs,  other  computing  devices,  and 
networks,  it  is  still  a  challenge  to  understand  the  implications  of  anom¬ 
alous  behavior  that  may  be  detected  by  intrusion  detection  systems, 
network  management  systems,  or  other  lA  tools.  In  some  cases,  only 
highly  trained  operators  can  safely  ignore  or  quickly  act  on  an  anoma¬ 
lous  reading. 


CHAPTER  THREE 


Overview  of  the  Current  DoD  Information 
Assurance  Certification  and  Accreditation  Process 


In  July  2006,  the  Assistant  Secretary  of  Defense  for  Networks  and 
Information  Integration/DoD  Chief  Information  Ofhcer  (ASD[NII]/ 
DoD  CIO)  signed  a  memorandum  establishing  an  interim  policy  for 
conducting  C&A  on  DoD  ISs  (Grimes,  2006).  The  memo  was  accom¬ 
panied  hy  a  guidance  document  that  outlined  the  specihc  procedure  to 
he  used  for  conducting  the  C&A  process  for  all  DoD  systems. 


DIACAP  Activities  and  Scope 

In  November  2007,  DoDI  8510.01,  “DoD  Information  Assurance  Cer¬ 
tification  and  Accreditation  Process  (DIACAP),”  established  as  DoD 
policy  the  guidance  included  in  the  2006  memorandum.  The  instruc¬ 
tion  outlines  a  series  of  individual  activities  or  tasks  that  are  to  be  exe¬ 
cuted  during  DoD  IS  C&A,  shown  in  Figure  3.1.  The  purpose  of  the 
DIACAP  is  to  provide  a  framework  for  implementing  required  or  nec¬ 
essary  lA  capabilities  and  services  and  an  auditable  and  transparent 
process  for  assessing  compliance. 

Each  of  the  steps  and  information  requirements  are  supported  by 
DoD  or  other  federal  policies  and  are  intended  to  parallel  various  steps 
in  the  life  cycle  of  a  particular  system,  beginning  with  initial  concep¬ 
tion  and  design  and  continuing  through  decommissioning. 
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Figure  3.1 
DIACAP  Activities 


Decommission 

system 
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•  Register  system  with  DoD 
component  lA  program 

•  Assign  lA  controls 

•  Assemble  DIACAP  team 

•  Initiate  DIACAP 
implementation  plan 


Implement  and  validate 
assigned  lA  controls 


Maintain  authorization  to 
operate  and  conduct  reviews 


•  Maintain  situational 
awareness 

•  Maintain  lA  posture 

•  Conduct  reviews  (review 
of  lA  controls  must  occur 
at  least  annually) 

•  Initiate  reaccreditation 


*  Execute  DIACAP 
implementation  plan 

»  Conduct  validation  activities 

*  Prepare  POA&M 

»  Compile  validation  results 
in  DIACAP  scorecard 


Make  certification  determination 
and  accreditation  decision 


•  Make  certification  determination 

•  Issue  accreditation  decision 


SOURCE:  DoDI  8510.01  (2007,  p.  13,  Figure  F.2). 
NOTE:  POA&M  =  Plan  of  Action  and  Milestones. 
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The  scope  of  DoDI  8510.01  includes  all  DoD-owned  and  -con¬ 
trolled  ISs  that  are  operated  by  a  contractor  or  other  entity  on  behalf  of 
DoD,  regardless  of  sensitivity  or  classification.' 


Definition  of  a  DoD  Information  System 

A  DoD  IS  is  defined  in  Department  of  Defense  Directive  (DoDD) 
8500. OlE  (2007,  para.  E2.1.16)  as  the 

[s]et  of  information  resources  organized  for  the  collection,  stor¬ 
age,  processing,  maintenance,  use,  sharing,  dissemination,  dis¬ 
position,  display,  or  transmission  of  information.  Includes  AIS 
applications,  enclaves,  outsourced  IT-based  processes,  and  plat¬ 
form  IT  interconnections. 


'  DoDI  8510.01  does  not  alter  or  supersede  existing  policies  or  authorities  associated  with 
sensitive  compartmentalized  information  or  special-access  programs. 
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Importantly,  Enclosure  E  of  DoDD  8500. OlE  does  not  preclude 
this  definition  from  being  applied  to  an  aggregation  of  multiple  ISs  or 
an  SoS.  In  other  words,  the  definition  of  DoD  IS  does  not  assert  that 
the  information  resources  specified  are  tied  to  a  particular  function 
or  mission,  but  rather  to  the  handling  and  manipulation  of  informa¬ 
tion.  Thus,  this  definition  does  not  restrict  the  number  or  types  of  IT 
resources  that  can  be  included  in  a  DoD  IS  and  evaluated  as  part  of  a 
single  C&A  assessment. 

However,  as  we  discuss  in  Chapter  four,  specific  steps  associated 
with  particular  DIACAP  activities  do  inhibit  or  prevent  a  single  C&A 
assessment  for  an  aggregation  of  multiple  ISs  or  an  SoS. 


DIACAP  Validation  Activities  and  Results 

In  the  interest  of  brevity,  we  do  not  describe  in  detail  the  steps  of  the 
DIACAP  process  in  this  monograph.  However,  in  this  section,  we 
highlight  a  few  important  points  regarding  DIACAP  validation  activi¬ 
ties.  A  key  part  of  the  validation  process  is  the  verification  that  the 
proper  information  assurance  controls  (lACs)  that  are  needed  to  main¬ 
tain  the  IS’s  assigned  MAC,  CE,  and  MC  levels  are  in  place,  prop¬ 
erly  designed,  and  work  effectively.  In  some  cases,  a  particular  system’s 
lACs  are  inherited  from  other  ISs  using  the  concept  of  inheritance 
described  in  DIACAP  policy  and  in  the  DIACAP  handbook  (U.S. 
Department  of  the  Navy,  2008). 

The  DIACAP  implements  the  lACs  identified  in  DoDI  8500.2, 
which  include  a  mandatory  set  of  controls  based  on  an  individual 
system’s  MAC  and  CE.  The  MAC  and  CE  are  independent  of  each 
another,  so  there  are  a  total  of  nine  possible  combinations.  The  MAC 
lA  controls  address  integrity  and  availability,  while  the  CE  lA  controls 
primarily  address  confidentiality. 

The  results  are  compiled  into  the  DIACAP  Scorecard,  which 
summarizes  the  lA  posture  of  an  individual  IS,  documents  the  accredi¬ 
tation  decision,  and  contains  a  listing  of  all  lACs  and  their  status.  The 
status  of  each  lAC  is  determined  by  one  or  more  validation  procedures. 
Some  of  these  procedures  may  include  tests  against  known  vulnerabili- 
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ties  (for  example,  computer  viruses).  The  status  of  each  lAC  is  indi¬ 
cated  on  the  scorecard  as  follows: 

Compliant  ( C).  lACs  for  which  expected  results  for  all  associated 
validation  procedures  have  been  achieved. 

Non-Compliant  (NC).  lACs  for  which  one  or  more  expected 
results  for  all  associated  validation  procedures  are  not  achieved. 

Not  achieving  expected  results  for  all  validation  procedures  does 
not  necessarily  equate  to  unacceptable  risk. 

Not  Applicable  (NA).  lACs  that  do  not  impact  the  security  posture 
of  the  IS  as  determined  by  the  [designated  accrediting  authority]. 

(U.S.  Department  of  the  Navy,  2009,  para.  4.4. 2. 7) 

Validation  tests  for  each  lAC  are  typically  conducted  using  test 
scripts  that  cover  a  wide  range  of  potential  vulnerabilities  and  have 
been  developed  in  prior  test  plans.  Test  results  are  compiled  and  cat¬ 
egorized  using  vulnerability,  weakness,  and  exposure  criteria  for  that 
particular  IS.  Later  in  this  monograph,  we  examine  how  well  such  tests 
plans,  reports,  and  results  can  be  aggregated  across  separate  and  pos¬ 
sibly  dissimilar  ISs. 


CHAPTER  FOUR 


Aggregation  Approach  to  DoD  Information 
Assurance  Certification  and  Accreditation 


Degrees  of  Aggregation 

The  degree  to  which  one  could  aggregate  DoD  ISs  for  the  purposes  of 
conducting  C&A  is  marked  by  two  endpoints.  At  one  end,  there  is  no 
aggregation  of  independently  developed  DoD  ISs.  This  endpoint  most 
closely  approximates  the  current  situation,  in  which  each  individual  IS 
is  documented,  reviewed,  and  assessed  independently  of  all  other  DoD 
ISs  with  which  it  may  interact. 

At  the  other  extreme  is  the  notion  of  aggregating  all  DoD  ISs 
associated  with  a  particular  site  or  platform,  which  we  refer  to  as  “full 
aggregation.”  While  it  is  conceptually  possible  to  aggregate  all  DoD 
ISs  across  a  department  or  service,  for  the  purposes  of  this  monograph, 
complete  or  full  aggregation  is  limited  to  DoD  systems  that  are  colo¬ 
cated  or  operate  on  a  common  platform. 

Between  these  extremes  is  the  notion  of  partial  aggregation  of 
DoD  ISs.  There  are  multiple  categorizations  that  could  be  used  for 
aggregating  “similar”  or  compatible  DoD  ISs.  It  may  be  possible  to 
combine  DoD  ISs  that  are  related  to  the  same  function  or  based  on 
specific  characteristics,  such  as  CL  or  MAC.'  We  evaluated  the  partial 
aggregation  scheme  that  assumed  that  DoD  ISs  would  be  aggregated 
based  on  three  separate  categories,  MAC,  CL,  and  MC,  as  shown 
in  Figure  4.1.  Each  category  contained  three  values.  In  other  words, 
a  particular  platform  or  location  would  have,  at  most,  27  DoD  ISs 
after  aggregation  was  applied.  These  three  categories  were  selected  to 
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See  Appendix  B  for  definitions  of  these  three  characteristics  and  their  levels. 
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Figure  4.1 

Hypothetical  Categories  for  Aggregating  DoD  Information  Systems 


NOTE:  "Public"  confidentiality  levels  refer  to  systems  whose  purpose  is  to  provide 
publicly  released  information  to  the  public  (DoDI  8500.2,  2003). 

RAND  MG9S1-4.1 

illustrate  the  concept  of  partial  aggregation  because  both  CL  and  MAC 
define  the  set  of  IS  lA  controls  that  must  be  implemented,  and  MC 
relates  to  the  role  of  the  aggregate  DoD  IS  relative  to  the  platform’s  or 
installation’s  mission  or  function. 

Once  the  systems  are  aggregated  or  combined  for  lA  C&A,  the 
number  of  systems  requiring  individual  C&A  will  typically  be  lower 
than  the  total  number  of  systems  on  the  platform.  This  form  of  aggre¬ 
gation  has  the  potential  to  produce  significant  efficiencies  (because  the 
number  of  individual  lA  tests  and  analyses  can  be  reduced  to  corre¬ 
spond  to  the  number  of  aggregated  ISs),  and  the  probability  of  detecting 
and  resolving  cross-IS  or  networked  IS  vulnerabilities  can  be  increased 
(presumably  because  a  comprehensive  set  of  network  vulnerability  tests 
can  be  run  against  all  ISs  at  once  in  each  aggregation). 

In  the  partial  aggregation  scheme  suggested  here,  the  final  number 
of  aggregated  DoD  ISs  that  require  individual  lA  C&A  will  depend 
on  the  number  of  categories,  the  degree  to  which  individual  systems 
can  be  combined,  and  the  specific  details  of  the  platform  or  location. 
Some  platforms  may  possess  few  distinct  types  of  DoD  ISs,  resulting 
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in  relatively  few  aggregate  systems  (each  aggregate  may  contain  a  large 
number  of  individual  ISs).  Alternatively,  a  fixed  site  or  carrier  that  con¬ 
tains  multiple  networks  and  supports  a  broad  range  of  missions  and 
functions  could  have  all  27  separate  aggregate  DoD  ISs.  However,  in 
practice,  it  would  be  uncommon  for  a  DoD  IS  to  have  a  MAC  level 
of  1  (i.e.,  a  system  that  is  vital  for  operational  readiness  or  mission 
effectiveness)  and  also  have  a  CL  of  “public”  and  an  MC  category  of 
“mission  support.”  Therefore,  for  most  platforms  and  locations,  there 
would  be  fewer  than  27  separate  aggregated  DoD  ISs. 

Using  partial  aggregation,  as  opposed  to  full  aggregation,  will 
require  a  systems  engineering  approach  for  defining  the  boundaries  of 
the  aggregated  DoD  IS,  identifying  the  interdependencies  between  the 
various  aggregates,  and  appropriately  applying  the  C&A  framework. 

We  reviewed  each  of  the  individual  DIACAP  activities  to  deter¬ 
mine  whether  any  current  DoD  or  federal  policies  inhibited  conducting 
the  C&A  process  for  DoD  ISs  that  are  fully  aggregated  (option  1)  or 
partially  aggregated  (option  2).  A  summary  of  those  findings  is  shown  in 
Table  4.1.  Option  3  is  intended  to  reflect  the  current  status  quo,  in  which 
no  aggregation  is  implemented  and  each  individual  DoD  IS  is  assessed 
and  certified  separately.  Green  cells  in  the  table  identify  DIACAP  activi¬ 
ties  for  which  no  policy  issues  were  identified. 

Yellow  indicates  DIACAP  activities  that  do  not  have  any  specific 
DoD  or  federal  policy  issues  but  could  be  difficult  to  implement  or 
accomplish  as  currently  described  in  DoDI  8510.01.  In  the  cases  of  full 
(option  1)  and  partial  (option  2)  aggregation  of  DoD  ISs,  the  difficul¬ 
ties  are  generally  related  to  the  added  complexity  incurred  as  a  result  of 
the  aggregation. 

Cells  that  are  shaded  red  indicate  DIACAP  activities  for  which 
clear  policy  issues  exist  that  would  inhibit  or  prevent  the  C&A  process 
as  written  from  being  applied  to  an  aggregated  DoD  IS. 

For  the  purposes  of  this  monograph,  we  discuss  only  those 
DIACAP  activities  for  which  specific  policy  issues  or  implementation 
challenges  were  identified. 
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Table  4.1 

Assessment  of  Policy  Issues  Related  to  IS  Aggregation 


Full  Partial  No 

Aggregation  Aggregation  Aggregation 

Degree  of  Aggregation  Option  7  Option  2  Option  3 


1.  Initiate  and  plan  lA  C&A 

-Register  system  with  DoD 
component  lA  program 

-Assign  lA  controls 

-Assemble  DIACAPteam 

-Initiate  DIACAP  implementation 
plan 

2.  Implement  and  validate  assigned 
lA  controls 

-Execute  DIACAP  implementation 
plan 

-Conduct  validation  activities 

-Prepare  POA&M 

-Compile  validation  results  and 
DIACAP  Scorecard 

3.  Make  certification  determination 
and  accreditation  decisions 

-Make  certification  determination 

-Issue  accreditation  decision 

4.  Maintain  authorization  to  operate 
and  conduct  reviews 

-Maintain  situational  awareness 

-Maintain  lA  posture 

-Conduct  review  (at  least  annually) 

-Initiate  reaccreditation 

5.  Decommission 

-Retire  system 

n  No  policy  issues  identified 
□  No  policy  issues  identified;  potential 

difficulties  with  implementation  identified 
B  Potential  policy  issue(s)  identified 
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Potential  DIACAP  Policy  issues 

Initiate  and  Plan  Information  Assurance  Certification  and 
Accreditation 

As  part  of  the  step  to  register  the  system  with  the  DoD  component 
lA  program  under  “Initiate  and  plan  lA  C&A”  (see  Table  4.1),  the 
DIACAP  requires  that  each  DoD  IS  being  reviewed  have  a  SIP.  Accord¬ 
ing  to  DoDI  8510.01  (2007),  the  SIP,  which  contains  32  unique  data 
elements,  is  used  throughout  the  DIACAP.  Attachment  1,  Enclosure 
E  of  DoDI  8510.01  includes  specific  instructions  along  with  the  mini¬ 
mum  data  requirements  for  identifying  a  particular  system.  The  SIP’s 
current  structure  prevents  option  1,  or  full  SoS  aggregation  of  DoD 
ISs.  Specific  data  elements  of  the  SIP  require  that  a  program  have  a 
unique  system  type  (e.g.,  AIS,  enclaves,  platform  IT  interconnection, 
outsourced  IT-based  processes),  that  it  be  associated  with  a  single  gov¬ 
erning  mission  area  (e.g.,  enterprise  information  environment,  busi¬ 
ness,  warfighting,  defense  intelligence),  and  that  it  also  have  a  single 
acquisition  phase  (i.e.,  pre-A,  A,  B,  C,  or  post-full-rate  production). 
The  current  SIP  record  format  also  requires  that  each  DoD  IS  have  a 
single,  unique  MC,  MAC,  or  CE.  In  addition,  only  a  single  accredita¬ 
tion  status  (i.e.,  ATO,  interim  ATO  [lATO],  interim  authorization  to 
test  [lATT],  or  denied  ATO  [DATO])  can  be  issued  to  a  system.  The 
complete  list  of  the  SIP  data  elements  identified  in  DoDI  8510.01  are 
presented  in  Appendix  A. 

The  requirement  that  these  SIP  data  elements  have  unique  entries 
(i.e.,  only  a  single  response  per  data  element,  not  multiple  entries  per 
element)  would  prevent  the  DIACAP  from  being  applied  to  a  DoD 
IS  that  was  aggregated  at  the  site  or  platform  level.  Most  platforms 
(e.g..  Navy  vessels)  have  many  DoD  ISs  at  different  MACs,  at  different 
CEs,  and  of  different  types.  A  single  aggregated  IS  that  included  all 
the  DoD  ISs  contained  therein  would  then  be  composed  of  multiple 
MACs,  multiple  CEs,  multiple  MC  levels,  and  perhaps  even  ISs  at  dif¬ 
ferent  phases  of  acquisition.  Such  a  system  could  not  be  registered  to 
perform  C&A  aggregation  using  the  SIP  structure  specified  in  current 
DoD  policy. 
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For  some  of  the  SIP  data  elements,  it  may  be  possible  to  set  the 
value  for  the  aggregated  IS  default  to  the  highest  or  most  restrictive 
level  contained  within  the  aggregate.  In  other  words,  if  an  aggregated 
DoD  IS  contained  public,  unclassified,  and  classified  ISs,  the  aggre¬ 
gate  would  need  to  meet  all  the  assigned  lA  controls  corresponding  to 
the  highest  CL  present.  However,  in  practice,  this  would  entail  having 
low-priority,  nonessential  ISs  held  to  the  same  standards  as  high-value, 
mission-essential  ISs.  This  would  not  only  impose  restrictive  lA  con¬ 
trols  on  ISs  that  were  never  intended  to  operate  under  those  conditions, 
potentially  causing  them  to  cease  to  function,  but  it  would  also  elimi¬ 
nate  the  justification  for  assigning  separate  lA  controls  based  on  MAC 
and  CL. 

Another  drawback  to  full  aggregation,  as  mentioned  earlier,  is 
the  assignment  of  a  single  accreditation  status.  Full  aggregation  would 
imply  that  if  a  single  nonessential,  non-mission-critical  system  on  a 
platform  or  at  a  common  location  were  issued  DATO,  the  entire  aggre¬ 
gate  DoD  IS  would  default  to  the  same  status. 

Implement  and  Validate  Information  Assurance  Controls 

One  of  the  steps  under  “Implement  and  validate  lA  controls”  (see 
Table  4.1)  requires  that  each  DoD  IS  be  included  in  the  department’s 
POA&M.  According  to  the  Federal  Information  Security  Management 
Act  (44  U.S.C.  3541  et  seq.)  and  Office  of  Management  and  Budget 
(OMB)  policy,  agencies  are  required  to  report  quarterly  on  IT  secu¬ 
rity  performance  measures  and  progress  toward  addressing  or  resolv¬ 
ing  known  IT  security  issues  through  the  POA&M  (Bolten,  2004). 
The  policy  also  explicitly  states  that  it  includes  both  non-national  secu¬ 
rity  programs  and  national  security  systems  and  programs.  According 
to  OMB  Memorandum  M-04-25  (Bolten,  2004,  p.  14),  an  agency’s 
POA&M  must  “[b]e  tied  to  the  agency’s  budget  submission  through 
the  unique  project  identifier  of  a  system.  This  links  the  security  costs 
for  a  system  with  the  security  performance  of  a  system.”  In  other  words, 
the  purpose  of  this  requirement  is  to  enable  the  monitoring  and  assess¬ 
ment  of  the  relationship  between  the  cost  of  information  security  and 
lA  controls  for  a  particular  system  and  that  system’s  lA  performance. 
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If  DoD  ISs  are  aggregated  fully  to  the  site  or  platform  level,  the 
aggregate  DoD  IS  would  not  match  the  corresponding  budget  submis¬ 
sions  provided  to  OMB,  unless  the  budget  for  all  the  ISs  at  a  single  site 
or  platform  were  similarly  aggregated  and  reported  to  OMB  as  such. 
Assuming  that  the  purpose  of  linking  information  security  spending 
with  security  performance  is  to  improve  transparency  and  accountabil¬ 
ity,  aggregating  all  DoD  ISs  to  the  platform  or  site  level  may  also  be  at 
odds  with  this  POA&M  reporting  requirement. 

In  addition,  the  POA&M  does  not  contain  detailed  descriptions 
of  specihc  weaknesses,  but  it  does  provide  sufficient  data  to  permit 
oversight  and  tracking.  Aggregation  to  the  site  or  platform  level  for  all 
information  security  weaknesses  will  likely  make  tracking  and  provid¬ 
ing  oversight  more  complex  and  opaque. 

Another  important  step  under  the  “Implement  and  validate  lA 
controls”  area  of  the  DIACAP  is  to  conduct  validation  activities.  These 
validation  activities  may  include  the  review  of  design  information  for 
the  IS  and  particular  lACs,  as  well  as  actual  lA  vulnerability  tests  using 
carefully  documented  test  scripts  that  cover  the  full  range  of  vulner¬ 
abilities  to  which  a  system  may  be  subject.  The  results  of  these  tests  are 
used  to  evaluate  whether  a  particular  lAC  meets  the  DIACAP  criteria 
discussed  in  Chapter  Three.  If  these  lAC  test  results  are  to  be  combined 
to  produce  an  aggregated  test  score  for  an  aggregated  or  combined  set 
of  ISs,  these  results  would  have  to  be  summed  or  combined  according 
to  some  sort  of  quantitative  combinatorial  criteria.  This  implies  that  the 
test  results  for  different  ISs  must  be  comparable  and,  ideally,  represent 
the  same  security  assessment  factors.  In  other  words,  a  common  set  of 
security  metrics  is  needed  to  assess  risk  for  a  combined  set  of  ISs. 

Security  metrics  that  are  measurable  and  that  can  be  combined 
for  multiple  ISs  have  turned  out  to  be  a  nontrivial  technical  challenge 
for  the  IT  community.  A  number  of  initiatives  have  been  created  to 
address  shortcomings  in  IS  security  metrics,  including  the  following:^ 

Common  Vulnerabilities  and  Exposures  (CVE®) — common  vul¬ 
nerability  identifiers 


^  For  detailed  descriptions  of  these  initiatives,  see  Making  Security  Measurable  (2009). 
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Common  Weakness  Enumeration  (CWE™) — list  of  software 
weakness  types 

Common  Attack  Pattern  Enumeration  and  Classification 
(CAPEC™) — list  of  common  attack  patterns 

Common  Configuration  Enumeration  (CCE™) — common  secu¬ 
rity  configuration  identifiers 

Common  Platform  Enumeration  (CPE™) — common  platform 
identifiers 

Center  for  Internet  Security  (CIS)  Consensus  Security  Metrics 
Definitions — set  of  standard  metrics  and  data  definitions  that 
can  be  used  across  organizations  to  collect  and  analyze  data  on 
security  process  performance  and  outcomes 

Twenty  Most  Important  Controls  and  Metrics  for  Effective  Cyber 
Defense  and  Continuous  [Federal  Information  Security  Manage¬ 
ment  Act]  Compliance — twenty  key  actions  or  security  “con¬ 
trols”  that  organizations  must  take  to  block  or  mitigate  known 
and  reasonably  expected  attacks 

SANS  [Institute]  Top  Twenty — SANS/FBI  consensus  list  of  the 
Twenty  Most  Critical  Internet  Security  Vulnerabilities  that  uses 
[common  vulnerability  and  exposure  identifiers]  to  identify  the 
issues 

OWASP  Top  Ten — ten  most  critical  Web  application  security 
flaws 

[Web  Application  Security  Consortium]  Web  Security  Threat 
Classification — list  of  Web  security  threats 

Open  Vulnerability  and  Assessment  Eanguage  (OVAE®) — 
standard  for  determining  vulnerability  and  configuration  issues 

Common  Result  Format  (CRF™) — standardized  assessment 
result  format  for  conveying  findings  based  on  common  names 
and  naming  schemes 

Common  Event  Expression  (CEE™) — standardizes  the  way  com¬ 
puter  events  are  described,  logged,  and  exchanged 
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Open  Checklist  Reporting  Language  (OCRL”) — standard  for 
creating  reports  used  in  compliance  evaluation.  (Making  Secu¬ 
rity  Measurable,  2009) 

The  significant  number  of  these  initiatives  indicates  the  complex¬ 
ity  and  the  scale  of  the  need  for  IS  vulnerability  testing  and  IS  design 
vulnerability  assessments.  A  potentially  even  greater  challenge  will  be 
to  develop  a  methodology  to  aggregate  the  results  that  complies  with 
emerging  common  IS  security  metrics  across  the  full  spectrum  of  met¬ 
rics  that  are  now  under  development.  For  these  reasons,  we  assessed  the 
“Conduct  validation  activities”  step  of  the  DIACAP  to  have  implemen¬ 
tation  challenges,  as  shown  in  Table  4.1. 

Decommission 

Current  guidance  in  DoDI  8510.01  (2007,  p.  22)  describes  the  vari¬ 
ous  steps  involved  in  decommissioning  a  DoD  IS.  These  steps  include 
removing  the  DIACAP  Scorecard  and  the  POA&Ms  from  all  tracking 
systems.  However,  DoDI  8510.01  does  not  provide  any  guidance  or 
options  for  decommissioning  or  modifying  only  portions  of  a  system. 
In  other  words,  for  a  DoD  IS  that  is  aggregated  to  include  all  DoD  ISs 
located  at  a  given  site  or  on  a  particular  platform  (i.e.,  option  1),  or  that 
is  aggregated  to  include  even  some  of  the  DoD  ISs  (i.e.,  option  2),  if 
any  IS  in  that  aggregation  needs  to  be  decommissioned  or  significantly 
modified,  then  the  entire  aggregated  system  must  either  be  decommis¬ 
sioned  or  lose  its  accreditation. 

In  the  DIACAP,  a  DoD  IS  is  treated  as  a  single  entity  and  char¬ 
acterized  by  the  various  attributes  defined  in  the  SIP.  Just  as  DoDI 
8510.01  does  not  allow  a  DoD  IS  to  have  more  than  one  lA  record 
type  (e.g.,  AIS  application,  enclave,  outsourced  IT-based  process,  plat¬ 
form  IT  interconnection)  or  more  than  one  MAC  level,  each  DoD  IS 
is  allowed  to  have  only  a  single  system  life-cycle  phase  (i.e.,  concept 
refinement,  technology  development,  system  development  and  demon¬ 
stration,  production  and  deployment,  operations  and  support,  disposal 
or  decommission). 
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Potential  DIACAP  Implementation  Difficulties  for 
Aggregate  Information  Systems 

Initiate  and  Plan  Information  Assurance  Certification  and 
Accreditation 

While  potentially  difficult,  a  partial  aggregation  approach  (option  2) 
could  he  applied  for  the  “Register  system  with  DoD  component  lA 
program”  activity  without  the  need  to  change  the  structure  of  the 
current  SIP  record.  The  difficulty  would  depend  on  which  categories 
were  used  for  the  aggregation  and  the  variation  of  the  DoD  IS  preset. 
Nonetheless,  a  partial  aggregation  strategy  would  still  require  careful 
thought  when  identifying  which  DoD  IS  to  aggregate,  as  well  as  con¬ 
sideration  of  the  interdependencies  and  information  exchange  between 
individual  systems  and  between  aggregates  of  DoD  ISs. 

A  full  aggregation  (option  1)  approach  would  also  be  difficult  to 
implement  for  the  “Assign  lA  controls”  activity  and  would  depend  on 
the  variation  in  MAC  and  CL  on  the  platform  or  location.  Based 
on  current  practices,  IS  lA  controls  would  default  to  the  highest  MAC 
and  CL  in  the  aggregate.  For  the  partial  aggregation  strategy  illus¬ 
trated  in  Table  4.1,  this  would  not  be  an  issue  because  all  the  systems 
with  similar,  MACs,  CLs,  and  MCs  would  be  combined  in  the  same 
aggregation.  However,  for  the  full  aggregation  approach,  public,  non- 
mission- essential  ISs  would  potentially  have  to  meet  the  same  stan¬ 
dards  as  classified,  mission-critical  systems.  While  it  is  conceivably 
doable,  the  process  of  assigning  lA  controls  would  be  more  difficult  if 
aggregated  to  the  entire  platform  or  location. 

Implement  and  Validate  Information  Assurance  Controls 

Under  the  “Prepare  POA&M”  activity,  a  partial  aggregation  approach 
(option  2)  could  potentially  achieve  the  appropriate  balance  between 
aggregation  and  traceability  of  the  program’s  lA  compliance.  Achiev¬ 
ing  both  aggregation  and  traceability  would  require  modihcation  to 
OMB  policy  and,  possibly,  to  DoD  policy.  While  the  task  of  rewriting 
OMB  and  DoD  policy  is  not  insurmountable,  OMB  policy  regarding 
POA&M  reporting  applies  to  all  federal  departments  and  agencies. 
Consequently,  changes  to  federal  policy  would  have  to  be  evaluated  to 
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determine  how  they  may  affect  other  departments  or  agencies.  There¬ 
fore,  changing  OMB  policy  to  suit  the  needs  of  a  single  department, 
such  as  DoD,  can  he  difficult,  particularly  if  the  same  need  is  not  per¬ 
ceived  across  the  rest  of  the  federal  government.  Even  if  the  changes 
can  he  made  to  OMB  policy,  it  will  likely  still  he  necessary  to  modify 
DoD  policies  for  lA  C&A,  which  can  also  he  challenging,  particularly 
when  the  policy  changes  have  the  potential  to  affect  large  portions  of 
the  DoD  program  baseline. 

The  “Compile  validation  results  and  DIACAP  Scorecard”  activ¬ 
ity  is  an  impediment  for  both  full  aggregation  (option  1)  and  partial 
aggregation  (option  2)  approaches.  This  step  involves  assessing  and 
compiling  compliance  data  for  each  assigned  lA  control.  Each  lA  con¬ 
trol  associated  with  a  particular  system  is  assigned  one  of  three  options: 
compliant,  noncompliant,  or  not  applicable.  Linder  the  current  score- 
card  method,  an  lA  control  is  attributed  to  the  entire  system,  and  it  can 
have  only  a  single  status  (i.e.,  compliant,  noncompliant,  or  not  appli¬ 
cable).  However,  under  an  aggregation  approach,  the  same  lA  control 
may  have  a  different  status  in  different  constituent  DoD  ISs  (i.e.,  one 
constituent  DoD  IS  may  be  “compliant”  while  another  is  “noncompli¬ 
ant”).  While  it  may  be  possible  for  every  I A  control  to  have  the  same 
status  across  the  aggregate  DoD  IS,  maintaining  the  visibility  and 
coherence  of  each  lA  control  across  and  entire  SoS  will  be  difficult. 
One  solution  would  be  to  monitor  and  track  each  component  lA  con¬ 
trol  separately.  A  partial  aggregation  approach  in  which  systems  are 
aggregated  by  MAC,  CE,  and  MC  (Figure  3.1)  would  make  this  activ¬ 
ity  less  difficult  compared  to  aggregating  DoD  ISs  of  different  MACs, 
CEs,  and  MCs  because  all  the  DoD  ISs  in  the  aggregate  would  have 
to  meet  the  same  lA  controls  and  have  approximately  the  same  level  of 
impact  on  the  mission. 

Make  Certification  Determination  and  Accreditation  Decisions 

The  “Make  certification  determination”  activity  requires  the  certify¬ 
ing  authority  to  assess  the  performance  of  the  lA  mechanisms  and  the 
system  behavior  in  the  greater  information  environment.  As  part  of 
this  activity,  the  certifying  authority  assigns  severity  categories  for  each 
particular  weakness  or  shortcoming  associated  with  a  given  DoD  IS. 
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The  final  DoD  IS  certification  determination  involves  consideration  of 
the  impact  codes  for  the  system,  which  is  in  indication  of  the  negative 
consequences  associated  with  failure  of  the  system,  and  the  severity 
categories,  which  relate  to  the  level  of  risk  with  an  identified  weak¬ 
ness  or  shortcoming  (e.g.,  a  specific  lA  control  that  is  noncompliant). 
According  to  DoDI  8510.01,  severity  codes  in  one  part  of  the  system 
may  be  offset  by  security  measures  in  place  in  a  separate  system  com¬ 
ponent.  However,  the  process  of  accounting  for  any  offset  that  one  lA 
mechanism  provides  for  another  is  difficult  under  the  best  of  circum¬ 
stances.  The  process  of  making  an  informed  certification  determina¬ 
tion  will  only  become  more  challenging  as  the  degree  to  which  DoD 
ISs  are  aggregated  increases  and  systems  become  both  larger  and  more 
interconnected. 

Once  the  certification  determination  is  complete,  the  designated 
accrediting  authority  issues  the  accreditation  decision.  According  to 
DoDI  8510.01,  only  a  single  accreditation  status  (i.e.,  ATO,  lATO, 
I  ATT,  DATO)  is  assigned  for  the  entire  system.  A  description  of 
each  accreditation  determination  and  its  relative  ranking  is  shown  in 
Table  4.2.  There  is  currently  no  process  or  method  for  documenting 
accreditations  at  the  component  level,  nor  are  there  any  methods  for 
combining  the  component  accreditations  to  determine  a  final  aggre¬ 
gate  accreditation  status. 

The  full  aggregation  (option  1)  approach  would  require  the  constit¬ 
uent  DoD  ISs  to  meet  lA  controls  associated  with  a  higher  CL  or  MAC 
level  than  those  for  which  they  were  initially  intended  or  designed,  thus 
making  the  process  of  achieving  an  ATO  for  the  aggregate  IS  more  dif¬ 
ficult.  However,  a  partial  aggregation  approach  (option  2)  in  which  all 
the  DoD  ISs  were  aggregated  by  MAC,  CL,  and  MC  would  provide 
a  more  manageable  solution  for  making  a  certification  determination 
and  issuing  an  aggregate  accreditation  decision  based  on  the  fact  that 
all  the  components  of  the  aggregated  system  would  be  designed  for,  and 
have  to  be  in  compliance  with,  the  same  set  of  lA  controls. 

Once  the  accreditation  decision  has  been  issued  for  each  of  the 
aggregated  DoD  ISs  in  a  particular  platform  or  location,  there  then 
must  be  a  specific  logic  or  clearly  defined  method  for  combining  the 
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Table  4.2 

Potential  DIACAP  Accreditation  Determination 


Accreditation 

Determination 

Description 

Relative 

Ranking 

ATO 

The  DoD  IS  is  authorized  to  process,  store,  or  transmit 
information,  granted  by  the  designated  accrediting 
authority.  Authorization  is  based  on  an  acceptable  lA 
design  and  implementation  of  assigned  lA  controls. 

Highest 

lATO 

The  DoD  IS  has  temporary  approval  to  operate, 
granted  by  the  designated  accrediting  authority, 
based  on  an  assessment  of  the  implementation  status 
of  the  assigned  lA  controls. 

Medium- 

high 

lATT 

The  DoD  IS  is  granted  temporary  approval  to  conduct 
system  testing  by  the  designated  accrediting  authority 
based  on  an  assessment  of  the  implementation  status 
of  the  assigned  lA  controls. 

Medium- 

low 

DATO 

The  DoD  IS  is  denied  authorization  to  operate  if  it  is 
determined  by  the  designated  accrediting  authority 
that  the  system  has  an  inadequate  lA  design  or  has 
failed  to  implement  assigned  lA  controls. 

Lowest 

SOURCE:  Bendel  (2006). 


individual  accreditation  decisions  into  a  single  accreditation  decision 
for  the  SoS.  Next,  we  present  a  potential  method  for  combining  the 
constituent  accreditation  decisions  to  achieve  a  platform  accreditation 
decision. 

C&A  Process  for  a  Notional  SoS.  The  example  is  intended  to  illus¬ 
trate  a  method  for  determining  the  accreditation  determination  for  a 
platform  that  contains  only  a  limited  number  of  specialized  DoD  ISs. 
In  our  example,  the  platform  contains  only  the  following  types  of  sys¬ 
tems,  identihed  by  MAC,  CL,  and  MC: 

•  MAC:  level  1 

•  CL:  classihed,  or  sensitive 

•  MC:  mission-critical,  or  mission-essential. 

Aggregating  all  of  the  ISs  by  these  three  categories  would  produce 
four  aggregate  DoD  ISs.  For  clarity,  the  four  aggregate  DoD  ISs  have 
been  labeled  “Weapons,”  “ISR”  (intelligence,  surveillance,  and  recon- 
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naissance),  “Nav”  (navigation),  and  “Comm”  (communication)  and  are 
shown  in  Table  4.3. 

The  first  step  involves  assessing  the  accreditation  determination 
for  each  aggregate  DoD  IS  independently.  This  could  be  done  using  a 
modified  DIACAP  C&A  procedure  that  incorporates  the  recommen¬ 
dations  presented  in  this  monograph. 


Table  4.3 

Example  Aggregate  DoD  IS  Interdependency  Matrix 
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After  the  accreditation  determination  has  been  established  for 
each  of  the  aggregate  DoD  ISs,  the  next  step  involves  assessing  the 
interdependency  of  each  aggregate  DoD  IS  relative  to  every  other 
aggregate  DoD  IS  on  the  platform  or  those  to  which  it  connects.  The 
four  aggregate  DoD  ISs  are  used  to  populate  the  sample  dependency 
matrix  shown  in  Table  4.3. 

For  this  particular  example,  as  shown  in  Table  4.4,  we  assigned 
three  levels  of  interdependency:  strong  (S),  weak  (W),  and  none  (blank). 
The  degree  of  interdependency  within  the  aggregate  will  be  unique  for 
each  platform  or  location  and  will  depend  on  the  specific  component 
DoD  ISs  present. 

In  the  table,  the  aggregate  for  “Weapons”  (row  1)  is  strongly  depen¬ 
dent  on  “ISR”  (column  B)  and  weakly  dependent  on  “Nav”  (column 
C).  The  relationship  between  aggregate  DoD  ISs  does  not  necessarily 
need  to  be  reciprocal.  The  DoD  IS  aggregate  “Weapons”  is  strongly 
dependent  on  “ISR”  (i.e.,  row  1  is  strongly  dependent  on  column  B), 
but  the  reverse  relationship  has  only  a  weak  dependency  (i.e.,  “ISR” 
is  only  weakly  dependent  on  “Weapons”).  The  term  dependent  means 
that  one  system  depends  on  the  output  or  data  of  another  system  to 
operate.  A  system  that  is  strongly  dependent  may  lose  complete  func¬ 
tionality  if  the  system  on  which  it  depends  ceases  to  function,  whereas 
a  system  that  is  weakly  dependent  would  lose  only  some  functionality. 

Once  the  relative  interdependencies  are  determined  and  the 
cells  in  Table  4.4  are  filled  in,  the  aggregate  DoD  ISs  are  assessed  to 
determine  how  their  relative  interdependencies  influence  their  overall 
accreditation  determination.  For  the  purposes  of  this  example,  two 
simple  logical  operations  are  constructed  to  illustrate  the  process: 

•  First,  if  one  aggregate  DoD  IS  is  strongly  dependent  on  a  second 
aggregate  DoD  IS  that  has  a  lower  accreditation  determination, 
then  the  inferior  accreditation  (based  on  the  relative  rankings 
shown  in  Table  4.2)  of  the  second  DoD  IS  is  assigned  to  the  first 
DoD  IS. 

•  Second,  if  one  aggregate  DoD  IS  is  only  weakly  dependent  on  a 
second  aggregate  DoD  IS,  then  the  accreditation  of  the  second 
aggregate  has  no  effect  on  the  first  unless  it  is  DATO,  in  which 
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case  the  first  aggregate  inherits  the  DATO  accreditation  from  the 
second  aggregate. 

We  used  these  two  rules  to  complete  the  overall  accreditation 
determination  for  each  aggregate  DoD  IS,  based  on  the  separate  accred¬ 
itation  determinations  and  the  relative  interdependencies,  as  shown  in 
column  E  in  Table  4.4. 


Table  4.4 

Example  Aggregate  DoD  IS  Interdependency  Matrix  with  Interdependent 
Accreditation  Determination 


Aggregation  Approach  to  DoD  lA  C&A  35 


The  application  of  this  method  to  an  actual  program  or  platform 
would  require  a  more  robust  set  of  interdependency  rules.  They  could 
potentially  be  designed  to  be  applicable  across  all  platforms  and  loca¬ 
tions.  Alternatively,  rules  could  be  developed  that  have  the  same  gen¬ 
eral  structure  but  could  also  be  tailored  to  suit  the  particular  mission 
or  function  for  the  platform  or  location  being  assessed. 

The  final  step  involves  using  the  interdependent  aggregate 
DoD  IS  accreditation  determinations  to  determine  a  final  platform 
accreditation.  The  process  described  in  our  example  is  illustrated  in 
Figure  4.2. 

A  process  for  combining  the  interdependent,  aggregate  DoD  IS 
accreditation  determinations  shown  at  step  3  in  Figure  4.2  would  need 
to  be  developed.  Like  the  rules  established  for  step  2,  they  could  be 
modified  or  tailored  to  the  particular  platform  or  mission. 

If  a  particular  platform  or  location  had  a  larger  number  of  aggre¬ 
gate  DoD  ISs,  the  matrix  shown  in  Tables  4.3  and  4.4  would  have  to 
be  scaled  up  to  include  rows  and  columns  for  each  aggregate  DoD  IS. 

Figure  4.2 

Schematic  Diagram  for  Platform  Accreditation  Determination  for 
Example  in  Table  4.4 
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This  also  opens  the  possibility  for  more  degrees  of  interdependency 
as  well  as  the  need  for  more  complex  rules  for  combining  the  various 
accreditation  determinations.  Nonetheless,  this  matrix  and  much  of 
the  process  could  be  automated  to  the  point  that  the  platform  program 
manager  inserts  the  separate  accreditation  determinations  for  each 
aggregate  DoD  IS  and  the  relative  interdependencies  into  a  spreadsheet 
or  tool.  The  rules  for  combining  could  then  be  either  user-defined  or 
predetermined  to  allow  the  user  see  an  overall  accreditation  determina¬ 
tion  for  the  platform  or  location,  similar  to  how  the  current  DIACAP 
Scorecard  operates.  As  mentioned  earlier,  this  method  would  also  allow 
changes  in  the  mission  or  accreditation  status  of  an  aggregate  DoD  IS 
to  be  incorporated  in  real  time  to  inform  the  platform  managers  of  the 
impact  on  mission  readiness  or  capability. 

Maintain  Authorization  to  Operate  and  Conduct  Reviews 

The  “Maintain  situational  awareness”  and  the  “Maintain  lA  posture” 
steps  represent  a  difficult  task  not  only  in  the  context  of  the  full  aggre¬ 
gation  and  partial  aggregation  approaches,  but  also  for  the  current 
status  quo.  In  all  three  cases,  the  highly  complex  interdependency  of 
DoD  ISs  means  that  integrating  new  systems,  updating  or  modifying 
existing  systems,  or  removing  or  decommissioning  antiquated  systems 
makes  maintaining  situational  awareness  and  lA  posture  a  daunting 
task,  even  under  the  best  of  circumstances. 

Similarly,  the  process  of  “Conducting  an  annual  review”  becomes 
more  difficult  as  the  level  of  aggregation  is  increased.  Conducting  a  full 
review  of  C&A  of  either  the  fully  aggregated  DoD  IS  (option  1)  or  the 
partially  aggregated  DoD  IS  (option  2)  will  require  more  time,  involve 
greater  interdependency,  and  potentially  have  a  greater  impact  on  the 
platform’s  readiness  to  operate. 


Balancing  Transparency  and  Reporting  Requirements 

Establishing  an  lA  aggregation  approach  that  balances  the  need  for 
transparency  and  reporting  requirements  is  a  difficult  task.  DoDI 
8510.01  requires  that  C&A  be  conducted  at  the  individual  DoD  IS 
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level.  However,  what  is  not  fully  accounted  for  in  the  instruction  is  the 
validity  or  fragility  of  that  accreditation  once  it  is  introduced  into  an 
SoS.  The  current  systems-only  approach  may  lead  to  lA  vulnerabili¬ 
ties  that,  while  not  present  when  the  system  is  assessed  independently, 
emerge  once  the  system  is  interconnected  with  other  DoD  ISs.  A  poten¬ 
tial  benefit  of  an  aggregated  lA  C&A  approach  is  that  it  may,  through 
an  SoS  approach,  explicitly  identify  interdependencies  between  aggre¬ 
gations  (in  the  case  of  a  partial  aggregation  strategy)  while  at  the  same 
time  reducing  the  total  number  of  individual  C&As  that  need  to  be 
performed. 

Another  important  reason  for  achieving  the  correct  balance  is  the 
ability  to  associate  an  organization’s  budget  for  a  specific  set  of  I A  capa¬ 
bilities  and  the  performance  of  those  capabilities.  However,  the  current 
POA&M  approach  required  by  OMB  associates  lA  cost  and  perfor¬ 
mance  with  a  system-specific  view.  It  assumes  that  if  the  lA  posture 
for  a  given  system  is  low,  the  program  may  not  be  provided  sufficient 
funds  for  lA  capabilities.  However,  recall  that  the  process  of  integrat¬ 
ing  (or  removing)  a  single  DoD  IS  into  (or  from)  a  larger  DoD  SoS  can 
be  a  source  of  lA  vulnerability.  Thus,  it  is  not  evident  that  the  sum  of 
funds  spent  on  lA  capabilities  for  individual  ISs  on  a  given  platform  or 
at  a  given  location  is  an  accurate  representation  of  the  overall  lA  in  the 
aggregate  system. 


Information  System  Information  Assurance  Pedigree 

A  question  related  to  lA  C&A  aggregation  is  how  program  managers 
should  handle  lA  certification  of  individual  systems  that  are  identi¬ 
cal  or  nearly  identical  internally  across  platforms — for  example,  across 
ships  of  the  same  ship  class  or  across  multiple  SoSs  or  network  imple¬ 
mentations.  Should  the  individual  IS  be  certified  for  each  instantia¬ 
tion  of  the  system,  regardless  of  the  similarity  or  difference  among  the 
systems’  operating  environments?  Repeating  lA  tests,  assessments,  and 
risk  estimates  for  the  same  IS  for  many  similar  platforms  and  SoSs 
or  network  implementations  may  increase  certification  cost,  perhaps 
substantially. 
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In  this  section,  we  discuss  the  concept  of  an  IS  lA  pedigree  and 
investigate  whether  it  can  provide  a  way  to  manage  and  control  the 
conhguration  of  individual  systems  that  are  part  of  an  lA  aggrega¬ 
tion  and  that  may  enable  program  managers  and  platform  managers  to 
reduce  the  cost  of  lA  certihcations,  especially  for  identical  systems  that 
are  implemented  across  ship,  aircraft,  or  other  platform  classes. 

The  Office  of  the  Deputy  Assistant  Secretary  of  Defense  for 
Cyher,  Identity,  and  Information  Assurance  has  issued  a  DoD  strategy 
that  provides  general  guidelines  for  building  trusted  platforms  and  sys¬ 
tems  that  can  be  used  to  develop  a  dehnition  of  lA  pedigree: 

“Pedigree”  of  a  piece  of  equipment — who  designed  it,  where  was 
it  built,  and  how  it  was  obtained 

“Pedigree”  of  a  system’s  life  cycle  -  who  installed  it  and  maintains 
it  and  how  is  its  security  parameters  configured  (e.g.,  best  com¬ 
mercial  practices,  DoD  configuration  guide,  federal  configura¬ 
tion  guide) 

Trust  level  of  the  entity  that  maintains  the  configuration  (e.g., 
industry  or  cleared  contractor/government  personnel) 

How  well  the  system  ensures  that  its  configuration  is  correct, 
e.g.,  are  software  downloads  limited  to  authorized  sources  and 
are  integrity  mechanisms  applied,  is  the  configuration  frequently 
checked  and  refreshed  from  known  good  source,  are  physical 
security  audits  performed,  and  are  inventory  controls  utilized? 
(ASD[NII]/DoD  CIO,  2009) 

These  guidelines  imply  that  an  IS  lA  pedigree  for  an  individual  IS 
should  contain  an  IS  lA  posture  assessment.  Such  a  posture  assessment 
can  be  designed  so  that  it  can  be  used  in  an  aggregated  lA  C&A  pro¬ 
cess  for  an  SoS  or  platform.  For  this  definition,  we  borrow  heavily  from 
the  IS  lA  pedigree  concept  under  development  by  Scott  Bell  in  his 
work  for  the  Office  of  the  Assistant  Secretary  of  the  Navy  for  Research, 
Development,  and  Acquisition,  Chief  Systems  Engineer.  However,  our 
definition  differs  in  some  significant  ways.  We  define  the  IS  lA  pedi¬ 
gree  as  containing  the  following  elements: 
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•  trusted  and  signed  IS  design  metadata  for  all  IS  hardware  and 
software  components 

•  IS  lA  control  profiles 

•  lA  test  scripts  and  plans 

•  lA  test  results 

•  external  interface  specifications,  including  DoDI  8510.01  (2007) 
compliance  verification  statements 

•  vulnerability  management  plan 

•  IS  configuration  management  plan. 

All  of  these  individual  items  are  specified  in  current  DoD  lA 
policy.  The  one  additional  item  that  is  of  special  interest  and  impor¬ 
tance  in  this  investigation  is  the  system  configuration  management 
plan.  This  plan  would  assure  IS  lA  managers  and  platform  and  SoS 
managers  that  the  configuration  of  the  IS  in  question  had  not  changed 
over  time  in  an  uncontrolled  way  and  that  the  latest  version  of  the 
IS  with  specified  lA  control  profiles,  configuration  changes,  or  system 
modifications  had  been  retested  and  recertified. 

It  is  important  to  point  out  that  the  definition  of  IS  lA  pedigree 
does  not  require  that  two  ISs  with  the  same  pedigree  have  exactly  the 
same  physical  configuration.  For  example,  the  first  IS  could  have  two 
PCs  connected  to  a  network  with  two  identical  Ethernet  ports,  while 
the  second  could  have  six  PCs  connected  to  a  network  with  six  identi¬ 
cal  Ethernet  ports.  These  two  ISs  could  be  determined  to  have  the  same 
pedigree  if  the  Ethernet  port  interface  specifications  were  identical. 

What  would  be  the  benefit  of  establishing  an  IS  lA  pedigree  for 
an  individual  IS  and  for  an  aggregated  approach  to  lA  certification? 
Underlying  our  analysis  of  lA  aggregation  is  the  assumption  that  indi¬ 
vidual  ISs  would  have  to  undergo  lA  certification  at  some  point  in  their 
development  cycle,  as  is  required  by  current  DoD  lA  policy.  However, 
once  an  individual  IS  has  been  certified  and  given  ATO  on  a  particular 
platform  or  in  a  particular  SoS,  this  lA  certification  could  be  applied 
to  the  same  type  of  IS  that  is  to  be  implemented  on  other  platforms  or 
in  other  SoSs.  The  IS  lA  pedigree  would  allow  this  implementation  to 
occur  without  a  second  recertification  of  the  individual  IS.  (An  aggre- 
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gated  IS  lA  certification  would  still  be  required  at  the  platform  or  SoS 
level.) 

The  pedigree  of  the  IS  could  be  maintained  only  if  the  program 
manager  could  demonstrate  that  the  configuration  of  the  system — all 
elements  of  the  lA  pedigree — were  unchanged  or,  if  changed,  had  been 
tested  and  recertified.  If  this  is  the  case,  then  the  system  would  be  given 
an  ATO  for  the  same  type  of  platform  or  SoS. 

For  example,  assume  that  a  particular  IS  is  certified  to  operate 
on  a  new  class  of  destroyer,  the  DDG-1000.  If  the  lA  pedigree  of  the 
IS  remained  unchanged,  then  it  could  be  implemented  on  later  ships 
of  the  same  ship  class  without  having  to  undergo  a  second  individual 
lA  certification.  However  the  second  implementation  of  the  IS  on  the 
second  destroyer  of  the  same  ship  class  would  still  have  to  undergo  an 
aggregated  lA  certification  (either  at  the  platform  level  or  at  a  lower 
partial  aggregation  level  for  a  particular  SoS  or  network). ^ 


^  Further  research  is  required  to  determine  whether  the  concept  of  IS  lA  pedigree  as  defined 
in  this  monograph  can  enable  an  effective  lA  C&A  aggregation  scheme  at  the  platform  level. 


CHAPTER  FIVE 


Observations  and  Recommended  Changes  to 
DoD  and  Federal  Policy 


In  the  current  DIACAP  framework,  DoD  ISs  are  handled  and  assessed 
individually.  There  is  justihcation  for  this  approach,  particularly  in  light 
of  the  POA&M  and  OMB’s  desire  to  he  able  to  link  IS  security  perfor¬ 
mance  with  IS  security  funding.  However,  the  current  C&A  process 
does  not  address  the  increasing  interdependency  of  these  systems  and 
the  potential  vulnerabilities  that  may  result  from  an  IS -centric  assess¬ 
ment.  Currently,  the  C&A  process  focuses  primarily  on  the  system 
with  only  limited  attention  given  to  the  connections  and  interdepen¬ 
dencies  between  the  various  systems,  which  can  be  a  source  of  vulnera¬ 
bility.  Similarly,  it  is  unclear  whether  the  DIACAP  can  be  applied  with 
conhdence  to  an  increasingly  decentralized  or  SOA  SoS.  Services  that 
are  developed  for  use  in  SOAs  typically  do  not  have  clear,  well-defined 
boundaries  that  comply  with  DoD  program  deliverables.  In  particular, 
DoD  is  attempting  to  maximize  reuse  of  data  and  software  services  by 
encouraging  the  development  of  SOA  services  that  can  be  employed 
across  system  boundaries  and  by  multiple  DoD  components  and  agen¬ 
cies.  These  services  are  being  designed  so  they  can  be  orchestrated  with 
other  services  and  applications  to  provide  highly  decentralized  capa¬ 
bilities  for  use  across  DoD.  What  does  it  mean  to  perform  C&A  on 
a  DoD  IS  or  capability  that  is  composed  of  multiple  services  that  are 
distributed  across  the  GIG  and  that  are  separately  owned,  controlled, 
certified,  and  accredited?  What  will  be  the  impact  on  DoD  ISs  that  are 
dependent  on  services  whose  underlying  lA  controls  or  operations  have 
changed?  It  is  not  clear  that  the  DIACAP  would  identify  or  provide 
adequate  guidance  for  addressing  these  issues. 
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In  addition,  it  is  known  from  systems  engineering  that  gaps  in 
desired  capabilities  or  functions  can  be  introduced  when  systems  are 
not  integrated  effectively.  This  has  led  to  an  increasing  emphasis  on  SoS 
integration  and  engineering.  An  idea  to  consider  is  whether  a  similar 
SoS  approach  needs  to  be  applied  to  lA  controls  and  the  C&A  process. 
In  the  same  way  that  adoption  of  systems  engineering  approaches  has 
led  to  improvements  in  the  overall  performance  and  function  of  sys¬ 
tems,  a  similar  emphasis  in  lA  may  lead  to  an  improved  level  of  lA  and 
protection  for  complex  SoS. 

The  appropriate  degree  of  aggregation  will  be  strongly  dependent 
on  the  platform  or  site,  as  well  as  its  function  or  mission.  If  the  scope 
of  aggregation  is  too  small,  it  will  resemble  the  current  state  of  affairs 
and  potentially  ignore  the  individual  connections  to  other  systems  that 
may  introduce  vulnerabilities.  However,  if  the  degree  of  aggregation  is 
too  large  (e.g.,  all  of  the  Internet),  the  aggregate  DoD  IS  will  be  too 
complex  to  assign,  certify,  and  accredit  reasonable  lA  controls. 

Our  preliminary  recommendations  focus  on  enabling  the  first 
steps  toward  aggregated  C&A  of  logical  collections  of  IT  and  national 
security  systems. 


Policy  Recommendations 

Restructure  the  SIP  and  the  DIACAP  Scorecard  to  enable  it  to  track  com¬ 
ponent  and  aggregate  DoD  ISs.  Current  policy  set  forth  in  DoDI  8510.01 
(2007)  requires  DoD  ISs  to  be  defined  by  single  attributes  (e.g.,  a  single 
MAC  level,  a  single  CL,  a  single  life-cycle  phase).  The  structure  and 
organization  of  the  SIP  would  need  to  be  modified  to  allow  collections 
of  systems  that  could  potentially  accommodate  multiple  DoD  ISs  in 
an  aggregated  DoD  IS.  It  would  also  need  to  be  modified  to  allow 
the  nesting  of  SIP  attributes  that  relate  to  either  individual  component 
DoD  ISs  or  higher-level  aggregated  DoD  ISs. 

Develop  an  acceptable  level  of  DoD  IS  aggregation  and  a  strategy 
for  tracking  IS  security  performance  between  POA&M  and  budget  doc¬ 
umentation.  One  of  OMB’s  primary  purposes  for  requiring  depart¬ 
ments,  services,  and  agencies  to  report  on  information  security  weak- 
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nesses  and  progress  toward  resolving  them  is  to  monitor  the  balance 
of  the  lA  and  information  security  budget  with  lA  performance.  It  is 
unlikely  that  this  requirement  will  change  in  the  near  future.  How¬ 
ever,  it  may  be  possible  to  work  with  OMB  to  understand  what  level  of 
granularity  is  needed  and  how  best  to  provide  information  that  is  both 
accurate  and  appropriate.  As  mentioned  earlier,  monitoring  individual 
lA  spending  and  capabilities  at  the  system  level  may  not  represent  the 
most  accurate  assessment  of  the  aggregate  lA  posture  at  the  platform 
(or  location)  level.  Intelligent  aggregation  of  DoD  IS  may  enable  more 
of  a  systems  engineering  approach  to  designing  and  implementing  lA 
controls,  which  could  potentially  lead  to  improved  lA  and  information 
security  performance  for  the  aggregate  DoD  IS. 

Develop  or  adopt  a  common  set  of  IS  security  metrics  that  can  be 
used  to  aggregate  lAC  validation  results  across  the  full  range  of  systems. 
Common  validation  result  reporting  mechanisms  that  can  be  mea¬ 
sured  using  existing  or  emerging  standards  are  needed  to  facilitate  the 
aggregation  of  test  and  design-review  assessment  results.  These  metrics 
will  need  to  be  applicable  across  the  spectrum  of  different  information 
systems  found  on  board  Navy  ships  and  platforms. 

Develop  specific  guidance  and  policy  for  modifying  or  decommis¬ 
sioning  a  component  or  subsystem  of  an  aggregated  DoD  IS.  Current 
policy  in  DoDI  8510.01  does  not  provide  any  information  or  guidance 
for  decommissioning  or  modifying  only  a  component  of  a  DoD  IS. 
Irrespective  of  developing  new  policies  for  C&A  of  aggregated  DoD 
ISs,  guidance  should  be  developed  and  promulgated  for  decommis¬ 
sioning  components  or  portions  of  existing  DoD  ISs.  As  mentioned 
in  Chapter  One,  a  DoD  IS  is  composed  of  multiple,  individual  IT 
resources — both  hardware  and  software.  In  most  cases,  the  compo¬ 
nent  IT  resources  could  be  defined  as  their  own  DoD  IS.  If  an  existing 
piece  of  hardware  or  software  that  is  integral  to  a  particular  DoD  IS  is 
removed,  replaced,  or  modihed  in  a  signihcant  way,  there  is  presently 
no  guidance  for  recertifying  or  reaccrediting  the  DoD  IS.  Therefore, 
developing  new  policy  for  C&A  of  DoD  ISs  in  which  components  or 
subsystems  are  decommissioned  or  modihed  will  not  only  beneht  cur¬ 
rent  DoD  systems;  it  will  also  be  useful  for  informing  program  manag- 
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ers  of  the  appropriate  methods  for  decommissioning  DoD  ISs  that  are 
part  of  a  larger,  aggregated  DoD  IS. 


Implementation  Recommendations 

Conduct  a  pilot  project  to  investigate  alternative  approaches  to  and  catego¬ 
ries  for  partial  aggregation,  and  to  assess  the  potential  benefits  oflA  controls 
and  C&A  procedures  for  an  aggregated  DoD  IS  or  DoD  SoS.  The  correct 
balance  of  partial  aggregation  for  a  DoD  IS  is  still  unknown.  Draw¬ 
ing  from  experience  in  other  areas  of  systems  engineering,  it  is  possible 
that  an  SoS  approach  to  lA  may  improve  the  overall  lA  posture  of  an 
SoS  and  enhance  overall  information  security  situational  awareness, 
lA  posture,  and  overall  performance  of  major  DoD  platforms,  such  as 
Navy  ships.  However,  this  has  yet  to  be  proven.  The  current  model  for 
lA  C&A  has  numerous  reporting  requirements  (i.e.,  one  DIACAP  for 
each  system),  even  for  systems  that  are  relatively  simple. 

In  addition,  the  current  IS  lA  C&A  framework  can  be  slow  and 
cumbersome  for  larger,  more  complex  ISs.  The  length  of  time  required 
to  certify  ISs  has  become  a  major  concern  and  a  “road  block”  that 
some  senior  military  commanders  have  chosen  to  bypass  entirely  to 
meet  urgent  operational  requirements  (Chiarelli,  2009).  Focusing  the 
lA  C&A  framework  at  an  appropriate  SoS  or  networking  level — as 
opposed  to  the  individual  system  level — could  eventually  benefit  the 
lA  certification  of  all  ISs  for  a  series  of  large  platforms,  such  as  a  Navy 
ship  class. 

An  approach  for  further  investigating  this  topic  would  be  to 
attempt  to  model  the  individual  DoD  systems  and  their  interactions  as 
part  of  an  aggregated  DoD  IS.  Potentially,  each  DoD  IS  could  be  mod¬ 
eled  as  an  independent  agent  with  inherent  attributes  (e.g.,  MAC,  CL, 
MC,  lA  controls)  and  an  explicitly  defined  interdependency  with  every 
other  system  on  the  platform.  It  may  be  possible  through  modeling  and 
simulation  to  determine  the  optimal  parameters  along  which  to  aggre¬ 
gate  separate  DoD  ISs.  It  may  also  provide  useful  information  about 
how  the  SoS  may  react  to  changes  in  individual  component  DoD  ISs 
due  to  modification,  decommission,  or  failure. 
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Another  approach  could  he  to  develop  a  pilot  project  whose  pur¬ 
pose  is  to  develop  modifications  to  the  current  C&A  procedures  and 
then  apply  them  to  aggregated  DoD  ISs  to  assess  information  security 
performance  compared  to  other  similar  systems  that  are  reviewed  and 
assessed  independently. 

Develop  a  definition  of  IS  lA  pedigree.  To  achieve  the  promise  of 
lA  C&A  aggregation  suggested  hy  Navy  officials  and  still  comply  with 
existing  DoD  lA  policy,  such  a  policy  may  need  to  he  supplemented  hy 
the  concept  of  an  IS  lA  pedigree,  perhaps  as  defined  in  this  monograph. 

An  important  related  question  is  whether  it  is  possible  to  reduce 
IS  lA  testing,  risk  analysis,  and  reporting  requirements  (i.e.,  reduce  the 
time  and  manpower  spent  conducting  C&A)  while  maintaining 
the  same  level  of  security  or  improving  it.  If  an  lA  aggregation  pilot 
study  were  conducted,  it  could  he  used  to  refine  the  concept  of  IS  lA 
pedigree.  However,  such  a  pilot  study  should  examine  how  the  concept 
of  IS  lA  pedigree  can  he  applied  to  a  second  SoS  or  platform,  after  an 
initial  IS  lA  pedigree  is  established.  The  second  case  in  the  pilot  study 
could  be  used  to  determine  whether  planned  lA  testing,  risk  analysis, 
and  reporting  requirements  are  reduced  and  by  how  much. 


A  Suggested  Partial  lA  Aggregation  Approach 

Based  on  the  initial  policy  analysis  presented  here,  one  approach  for 
aggregating  DoD  ISs  would  be  to  use  MAC,  CL,  and  MC  as  the  prin¬ 
cipal  categories  for  aggregation.  The  central  role  that  MAC,  CL,  and 
MC  play  in  assigning  specific  lA  controls,  assessing  system  risk  to  the 
mission,  and  identifying  potential  lA  weaknesses  or  gaps  suggests  that 
they  would  be  the  reasonable  candidates  for  an  initial  lA  aggregation 
approach.  Furthermore,  they  would  likely  require  relatively  few  mod¬ 
ifications  to  the  current  process  outlined  in  DoDI  8510.01.  Systems 
with  specific  MAC,  CL,  and  MC  categories  can  then  be  defined  using 
criteria  that  will  satisfy  C&A  requirements.  Systems  that  transcend 
across  categories  must  be  held  to  a  higher  standard  to  meet  the  same 
criteria. 
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The  current  DIACAP  process  has  been  characterized  as  a  signifi¬ 
cant  improvement  over  its  predecessor  by  some  of  its  authors  in  the  lA 
community.  However,  it  is  not  without  its  own  limitations,  and  some 
program  managers  reportedly  do  not  share  the  same  view  of  the  new 
process.  lA  managers  tasked  with  maintaining  lA  situational  aware¬ 
ness  and  preserving  the  current  lA  posture  struggle  at  times  to  fully 
comprehend  and  account  for  potential  lA  vulnerabilities  because  they 
are  limited  to  a  system- centric  perspective.  As  DoD  and  the  rest  of 
the  federal  government  move  toward  a  more  decentralized  and  service- 
oriented  architecture,  these  tasks  will  only  become  more  daunting,  and 
an  ever-increasing  number  of  potentially  critical  lA  vulnerabilities  will 
likely  go  unidentified  until  it  is  too  late.  Methods  for  conducting  C&A 
and  applying  lA  controls  will  need  to  be  reevaluated  and  updated  to 
provide  users  and  managers  with  the  appropriate  methods  for  ensur¬ 
ing  maximum  protection  and  assurance  of  their  information,  for  their 
platform  or  location,  and  across  the  GIG. 
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Table  A.l  presents  the  32  SIP  data  elements  included  in  Enclosure  3 
of  DoDI  8510.01  (2007).  In  July  2008,  the  U.S.  Department  of  the 
Navy’s  DIACAP  working  group  released  its  DIACAP  handbook  (U.S. 
Department  of  the  Navy,  2008),  which  included  a  table  describing  the 
various  elements  of  the  SIP.  While  the  SIPs  presented  in  both  docu¬ 
ments  are  similar,  there  are  notable  discrepancies.  For  instance,  the  SIP 
provided  in  the  Navy’s  DIACAP  handbook  contains  a  total  of  48  data 
elements,  compared  to  the  32  data  elements  described  in  DoDI  8510.01 
and  shown  in  Table  A.l.  Most  of  the  additional  data  elements  in  the 
Navy’s  DIACAP  handbook  request  additional  details  about  the  project 
or  are  Navy-centric.  However,  a  direct  comparison  between  the  two 
documents  also  reveals  that  there  is  not  a  one-to-one  match  between  all 
the  data  elements  in  the  DoDI  8510.01  SIP  and  in  the  Navy’s  DIACAP 
handbook.  Twenty-eight  of  the  32  SIP  data  elements  included  in  DoDI 
8510.01  are  reflected  in  the  handbook  SIP  data  elements,  although  in 
some  cases,  there  are  slight  differences  in  the  description  or  text. 

Two  of  the  data  elements  in  DoDI  8510.01  have  slightly  differ¬ 
ent  interpretations  in  the  Navy’s  DIACAP  handbook.  Specifically, 
“Governing  mission  area”  (ID  16  in  Table  A.l)  is  not  directly  pres¬ 
ent  in  the  SIP  included  the  handbook.  The  closest  matching  data  ele¬ 
ment  is  “Type  of  IT  investment”  (U.S.  Department  of  the  Navy,  2008, 
Enclosure  5,  Field  ID  33).  Similarly,  “Accreditation  status”  (ID  20  in 
Table  A.l),  which  seems  to  imply  the  current  accreditation  status  of  the 
DoD  IS,  is  not  present  in  the  SIP  in  the  Navy’s  DIACAP  handbook. 
The  closest  matching  data  element  appears  to  be  “Accreditation  request 
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type”  (U.S.  Department  of  the  Navy,  2008,  Enclosure  5,  Field  ID  28). 
However,  in  the  former  case,  the  data  element  indicates  the  current 
status  of  the  DoD  IS  being  assessed  (i.e.,  unaccredited,  ATO,  lATO, 
lATT,  or  DATO),  while  in  the  latter  it  is  an  indication  of  the  desired 
hnal  disposition  of  the  assessment  (i.e.,  ATO,  lATO,  interim  authority 
to  huild,  interim  authority  to  connect,  or  lATT). 

Finally,  two  of  the  SIP  data  elements  in  the  DoDI  8510.01  SIP 
do  not  appear  to  he  represented  in  the  Navy’s  DIACAP  handbook 
SIP.  The  first  is  “Accreditation  documentation”  (ID  22  in  Table  A.l), 
which  asks  whether  there  is  documentation  to  support  the  current 
accreditation.  The  second  is  “Privacy  Act  system  of  records  notice 
required”  (ID  26  in  Table  A.l),  which  asks  whether  a  Privacy  Act 
system  or  record  notice  is  required  per  DoD  Regulation  5400.1 1-R, 
“Department  of  Defense  Privacy  Program,”  (2007). 


Table  A.l 

SIP  Data  Elements  from  DoDI  8510.01,  Enclosure  3 


ID 


1 


2 


3 


4 

5 


6 

7 


Data  Element 
Descriptor 

Example,  Acceptable  Values, 
or  Comment 

Required  or 
Conditional^ 

System 

identification 

Provide  the  system  identification  number 
or  code  used  by  the  DoD  component  to 
uniquely  identify  the  system. 

Required/ 

system 

generated 

System  owner 

List  the  element  or  organization  in  the 

DoD  component  that  owns,  controls,  or 
manages  the  IS. 

Required 

Governing  DoD 
component  lA 
program 

List  the  DoD  component  that  owns 
the  IS. 

Required 

System  name 

Provide  the  full  descriptive  name  (e.g.. 
Agency  Billing  System). 

Required 

Acronym 

Provide  a  shortened  or  commonly  used 
name  or  abbreviation  (upper  case,  e.g., 
ABS). 

Required 

System  version  or 
release  number 

List  the  version  or  release  number  for  the 

IS  (e.g.,  1.0). 

Required 

System  description 

Provide  a  narrative  description  of  the 
system,  its  function,  and  uses.  Indicate 
whether  it  is  a  stand-alone  system. 

Required 
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Table  A.1 — Continued 


Data  Element  Example,  Acceptable  Values,  Required  or 

Descriptor  or  Comment  Conditional^ 


DIACAP  activity 

Identify  the  current  DIACAP  activity: 

1.  Initiate  and  plan  lA  C&A 

2.  Implement  and  validate  assigned  lA 
controls 

3.  Make  certification  determination  and 
accreditation  decision 

4.  Maintain  ATO  and  conduct  reviews 

5.  Decommission. 

Required 

System  life-cycle 
phase 

Identify  the  current  life-cycle  phase  of 
the  IS: 

1.  Concept  refinement 

2.  Technology  development 

3.  System  development  and 
demonstration 

4.  Production  and  deployment 

5.  Operations  and  support 

6.  Disposal  or  decommissioning. 

Required 

System  acquisition 
phase 

For  programs  of  record,  identify  the 

current  system  acquisition  phase: 

1.  Pre-Milestone  A  (concept  refinement) 

2.  Post-Milestone  A  (technology 
development) 

3.  Post-Milestone  B  (system  development 
and  demonstration) 

4.  Post-Milestone  C  (production  and 
deployment) 

5.  Post-full-rate  production/deployment 
decision. 

Conditional 

lA  record  type 

Identify  the  type  of  DoD  IS: 

1.  AIS  application 

2.  Enclave  (indicate  whether  stand-alone 
or  network  demilitarized  zone) 

3.  Outsourced  IT-based  process  (indicate 
whether  DoD-controlled  or  whether 
control  is  shared  with  service  provider) 

4.  Platform  IT  interconnection. 

Required 

Mission  criticality 

Identify  the  mission  criticality  (i.e., 
mission-critical,  mission-essential,  or 
mission  support). 

Required 

Accreditation 

vehicle 

Identify  the  C&A  process  that  was  or 
is  being  used  for  C&A  of  the  IS  (e.g., 
DIACAP;  Director  of  Central  Intelligence 
Directive  6/3,  1999;  NIST  800-37,  2004). 

Required 
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Table  A.1 — Continued 


Data  Element  Example,  Acceptable  Values,  Required  or 

Descriptor  or  Comment  Conditional^ 


14  Additional 
accreditation 
requirements 


15  Acquisition 
category 

16  Governing  mission 
area 


17  Software  category 

18  MAC  level 

19  Confidentiality 
level 

20  Accreditation 
status 


21  Certification  date 

22  Accreditation 
documentation 

23  Accreditation  date 


24  Authorization 
termination  date 


Identify  any  additional  accreditation 
requirements  beyond  the  lA  C&A  process 
(e.g.,  privacy,  special  access  requirements, 
cross-security  ciomain  solutions, 
Nonsecure  Internet  Protocol  Router 
Network,  Secret  Internet  Protocol  Router 
Network,  ports,  protocols,  and  services 
management). 

Identify  the  acquisition  category,  if 
applicable  (e.g..  Acquisition  Category  I). 

Identify  the  mission  area:  enterprise 
information,  environment,  business, 
warfighting,  or  defense  intelligence. 

Identify  whether  the  system  software  is 
COTS  or  government,  off  the  shelf. 

List  the  IS's  MAC  level  (i.e.,  MAC  I,  MAC  II, 
or  MAC  III). 

List  the  IS's  CL  (i.e.,  public,  sensitive,  or 
classified). 

Identify  the  accreditation  status  of  the 
IS  (i.e.,  unaccredited,  ATO,  lATO,  lATT,  or 
DATO). 

List  the  date  the  IS  was  certified  by  the 
certifying  authority. 

Are  there  documentation  and  artifacts 
that  support  the  accreditation  status? 

List  the  date  of  the  current  accreditation 
decision  (ATO,  lATO,  lATT,  or  DATO).  If 
the  IS  has  no  accreditation  determination, 
enter  "NONE"  and  the  projected 
accreditation  date. 

List  the  date  that  the  current 
accreditation  (ATO,  lATO,  or  lATT)  is  set 
to  expire. 


Conditional 


Conditional 


Required 


Required 


Required 


Required 


Required 
(default  is 
unaccredited) 

Conditional 


Conditional 


Required 


Conditional 
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Table  A.1 — Continued 


Data  Element  Example,  Acceptable  Values,  Required  or 

Descriptor  or  Comment  Conditional^ 


25  DIACAP  team  Identify  the  DIACAP  team  (e.g., 

roles,  member  designated  accrediting  authority, 
names,  and  contact  certifying  authority,  the  DoD  IS  program 
information  manager  or  system  manager,  the  DoD 

IS  lA  manager,  information  assurance 
officer,  user  representative. 


26  Privacy  impact  Indicate  whether  a  privacy  impact 

assessment  assessment  is  required  for  a  new  or 

required  previously  existing  IT  system. 


27  Privacy  Act  system  Indicate  whether  a  Privacy  Act  system  of 
of  records  notice  record  notice  is  required, 
required 


Required 


Required 


Required 


28  E-authentication  Indicate  whether  an  e-authentication  risk  Required 
risk  assessment  assessment  has  been  performed  according 
required  to  OMB  IVI-04-04  (Bolten,  2003). 


29  Date  of  annual  List  the  date  of  the  last  annual  security  Required 

security  review  review  for  systems  with  an  ATO.  Required 
for  ISs  with  an  ATO  in  effect  for  more 
than  one  year. 


30  System  operation  Identify  whether  the  system  operation  is  Required 

1.  Government  (DoD)-owned, 
government-operated 

2.  Government  (DoD)-owned,  contractor- 
operated 

3.  Contractor-owned,  contractor-operated 
(including  outsourced  IT  services) 

4.  Contractor-owned,  government  (DoD)- 
operated 

5.  Non-DoD  (including  federal,  state,  and 
local  governments;  grantees,  industry 
partners,  etc.). 


31  Contingency  plan 
required 


Indicate  whether  a  contingency  plan 
addressing  disruptions  in  operations  of 
the  IS  is  in  place. 


Required 


32  Contingency  plan 
tested 


Indicate  whether  the  contingency  plan 
that  is  in  place  has  been  tested. 


Required 


SOURCE:  DoDI  8510.01  (2007,  Enclosure  3). 

^  Required  entries  are  mandatory  for  completing  the  SIP.  Conditional  entries  must  be 
completed  if  they  apply  to  the  system  being  profiled.  If  the  entry  does  not  apply,  it 
should  be  left  blank. 
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Definitions  of  MAC,  CL,  and  MC 


Unless  otherwise  indicated,  the  following  definitions  are  reprinted  ver¬ 
batim  from  their  respective  source  documents. 

Mission  Assurance  Category.  “Applicable  to  DoD  information 
systems,  the  mission  assurance  category  reflects  the  importance  of 
information  relative  to  the  achievement  of  DoD  goals  and  objectives, 
particularly  the  warfighters’  combat  mission.  Mission  assurance  cate¬ 
gories  are  primarily  used  to  determine  the  requirements  for  availability 
and  integrity.  The  Department  of  Defense  has  three  defined  mission 
assurance  categories”  (DoDI  8500.2,  2003,  Enclosure  2,  p.  22). 

•  Mission  Assurance  Category  I  (MAC  I).  “Systems  handling  infor¬ 
mation  that  is  determined  to  be  vital  to  the  operational  readi¬ 
ness  or  mission  effectiveness  of  deployed  and  contingency  forces 
in  terms  of  both  content  and  timeliness.  The  consequences  of 
loss  of  integrity  or  availability  of  a  MAC  I  system  are  unaccept¬ 
able  and  could  include  the  immediate  and  sustained  loss  of  mis¬ 
sion  effectiveness.  Mission  Assurance  Category  I  systems  require 
the  most  stringent  protection  measures”  (DoDI  8500.2,  2003, 
Enclosure  2,  p.  22). 

•  Mission  Assurance  Category  II  (MAC  II).  “Systems  handling  infor¬ 
mation  that  is  important  to  the  support  of  deployed  and  contin¬ 
gency  forces.  The  consequences  of  loss  of  integrity  are  unaccept¬ 
able.  Eoss  of  availability  is  difficult  to  deal  with  and  can  only  be 
tolerated  for  a  short  time.  The  consequences  could  include  delay 
or  degradation  in  providing  important  support  services  or  com- 
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modities  that  may  seriously  impact  mission  effectiveness  or  opera¬ 
tional  readiness.  Mission  Assurance  Category  II  systems  require 
additional  safeguards  beyond  best  practices  to  ensure  assurance” 
(DoDI  8500.2,  2003,  Enclosure  2,  p.  22). 

•  Mission  Assurance  Category  III  (MAC  III).  “Systems  handling 
information  that  is  necessary  for  the  conduct  of  day-to-day  busi¬ 
ness,  but  does  not  materially  affect  support  to  deployed  or  con¬ 
tingency  forces  in  the  short-term.  The  consequences  of  loss  of 
integrity  or  availability  can  be  tolerated  or  overcome  without 
significant  impacts  on  mission  effectiveness  or  operational  readi¬ 
ness.  The  consequences  could  include  the  delay  or  degradation 
of  services  or  commodities  enabling  routine  activities.  Mission 
Assurance  Category  III  systems  require  protective  measures,  tech¬ 
niques,  or  procedures  generally  commensurate  with  commercial 
best  practices”  (DoDI  8500.2,  2003,  Enclosure  2,  pp.  22-23). 

Confidentiality  Level.  “Applicable  to  DoD  information  systems, 
the  conhdentiality  level  is  primarily  used  to  establish  acceptable  access 
factors,  such  as  requirements  for  individual  security  clearances  or  back¬ 
ground  investigations,  access  approvals,  and  need-to-know  determina¬ 
tions;  interconnection  controls  and  approvals;  and  acceptable  methods 
by  which  users  may  access  the  system  (e.g.,  intranet,  Internet,  wireless). 
The  Department  of  Defense  has  three  dehned  confidentiality  levels: 
classified,  sensitive,  and  public”  (DoDI  8500.2,  2003,  Enclosure  2, 

p.  16). 

Mission  Criticality,  Mission-Critical  Information  System.  “A 

system  that  meets  the  dehnitions  of ‘information  system’  and  ‘national 
security  system’  in  the  [Clinger-Cohen  Act],  the  loss  of  which  would 
cause  the  stoppage  of  warfighter  operations  or  direct  mission  sup¬ 
port  of  warhghter  operations.  (The  designation  of  mission  critical 
shall  be  made  by  a  Component  Head,  a  Combatant  Commander,  or 
their  designee.  A  hnancial  management  IT  system  shall  be  consid¬ 
ered  a  mission-critical  IT  system  as  dehned  by  the  Under  Secretary  of 
Defense  [Comptroller].)  A  ‘Mission-Critical  Information  Technol¬ 
ogy  System’  has  the  same  meaning  as  a  ‘Mission-Critical  Information 
System’”  (DoDI  5000.02,  p.  48,  Table  8). 
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Mission-Essential  Information  System.  “A  system  that  meets  the 
definition  of  ‘information  system’  in  Reference  (v),  that  the  acquir¬ 
ing  Component  Head  or  designee  determines  is  basic  and  necessary 
for  the  accomplishment  of  the  organizational  mission.  (The  designa¬ 
tion  of  mission-essential  shall  he  made  hy  a  Component  Head,  a  Com¬ 
batant  Commander,  or  their  designee.  A  financial  management  IT 
system  shall  be  considered  a  mission-essential  IT  system  as  defined  by 
the  [Under  Secretary  of  Defense  (Comptroller)].)  A  ‘Mission-Essential 
Information  Technology  System’  has  the  same  meaning  as  a  ‘Mission- 
Essential  Information  System’”  (DoDI  5000.02,  2008,  Table  8,  p.  48.). 

Mission-Support  Information  System.  If  the  information  system 
is  neither  mission-critical  nor  mission-essential,  it  is  labeled  mission  sup¬ 
port  (based  on  DoDI  8510.01,  2007,  p.  37,  Table  E3.A1.T1). 
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